<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.14: http://docutils.sourceforge.net/" />
<title>From Python to Numpy</title>
<meta content="An open-source book about numpy vectorization techniques, based on experience, practice and descriptive examples" name="description" />
<meta content="width=device-width, initial-scale=1, maximum-scale=1" name="viewport" />
<link rel="stylesheet" href="book.css" type="text/css" />
<link rel="stylesheet" href="math.css" type="text/css" />
</head>
<body>
<div class="document" id="from-python-to-numpy">
<h1 class="title">From Python to Numpy</h1>
<h2 class="subtitle" id="copyright-c-2017-nicolas-p-rougier-nicolas-rougier-inria-fr">Copyright (c) 2017 - Nicolas P. Rougier &lt;<a class="reference external" href="mailto:Nicolas&#46;Rougier&#37;&#52;&#48;inria&#46;fr">Nicolas<span>&#46;</span>Rougier<span>&#64;</span>inria<span>&#46;</span>fr</a>&gt;</h2>

<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<!-- Title:   From Python to Numpy -->
<!-- Author:  Nicolas P. Rougier -->
<!-- Date:    January 2017 -->
<!-- License: Creative Commons Share-Alike Non-Commercial International 4.0 -->
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<div class="title-logos docutils container">
<img alt="data/cc.large.png" src="data/cc.large.png" style="width: 40px;" />
<img alt="data/by.large.png" src="data/by.large.png" style="width: 40px;" />
<img alt="data/sa.large.png" src="data/sa.large.png" style="width: 40px;" />
<img alt="data/nc.large.png" src="data/nc.large.png" style="width: 40px;" />
<div class="line-block">
<div class="line"><br /></div>
<div class="line">Latest version - January 2019</div>
<div class="line">DOI: <a class="reference external" href="http://doi.org/10.5281/zenodo.225783">10.5281/zenodo.225783</a></div>
</div>
</div>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<div class="title-logos docutils container">
<img alt="data/cubes.png" src="data/cubes.png" style="width: 100%;" />
</div>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<p>There are already a fair number of books about Numpy (see <a class="reference internal" href="#bibliography">Bibliography</a>) and a
legitimate question is to wonder if another book is really necessary. As you
may have guessed by reading these lines, my personal answer is yes, mostly
because I think there is room for a different approach concentrating on the
migration from Python to Numpy through vectorization. There are a lot of
techniques that you don't find in books and such techniques are mostly learned
through experience. The goal of this book is to explain some of these
techniques and to provide an opportunity for making this experience in the
process.</p>
<p><strong>Website:</strong> <a class="reference external" href="http://www.labri.fr/perso/nrougier/from-python-to-numpy">http://www.labri.fr/perso/nrougier/from-python-to-numpy</a></p>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<div class="contents main-content topic" id="table-of-contents">
<p class="topic-title first"><strong>Table of Contents</strong></p>
<ul class="simple">
<li><a class="reference internal" href="#preface" id="id62">Preface</a><ul>
<li><a class="reference internal" href="#about-the-author" id="id63">About the author</a></li>
<li><a class="reference internal" href="#about-this-book" id="id64">About this book</a></li>
<li><a class="reference internal" href="#license" id="id65">License</a></li>
</ul>
</li>
<li><a class="reference internal" href="#introduction" id="id66">Introduction</a><ul>
<li><a class="reference internal" href="#simple-example" id="id67">Simple example</a></li>
<li><a class="reference internal" href="#readability-vs-speed" id="id68">Readability vs speed</a></li>
</ul>
</li>
<li><a class="reference internal" href="#anatomy-of-an-array" id="id69">Anatomy of an array</a><ul>
<li><a class="reference internal" href="#id5" id="id70">Introduction</a></li>
<li><a class="reference internal" href="#memory-layout" id="id71">Memory layout</a></li>
<li><a class="reference internal" href="#views-and-copies" id="id72">Views and copies</a></li>
<li><a class="reference internal" href="#conclusion" id="id73">Conclusion</a></li>
</ul>
</li>
<li><a class="reference internal" href="#code-vectorization" id="id74">Code vectorization</a><ul>
<li><a class="reference internal" href="#id7" id="id75">Introduction</a></li>
<li><a class="reference internal" href="#uniform-vectorization" id="id76">Uniform vectorization</a></li>
<li><a class="reference internal" href="#temporal-vectorization" id="id77">Temporal vectorization</a></li>
<li><a class="reference internal" href="#spatial-vectorization" id="id78">Spatial vectorization</a></li>
<li><a class="reference internal" href="#id19" id="id79">Conclusion</a></li>
</ul>
</li>
<li><a class="reference internal" href="#problem-vectorization" id="id80">Problem vectorization</a><ul>
<li><a class="reference internal" href="#id21" id="id81">Introduction</a></li>
<li><a class="reference internal" href="#path-finding" id="id82">Path finding</a></li>
<li><a class="reference internal" href="#fluid-dynamics" id="id83">Fluid Dynamics</a></li>
<li><a class="reference internal" href="#blue-noise-sampling" id="id84">Blue noise sampling</a></li>
<li><a class="reference internal" href="#id29" id="id85">Conclusion</a></li>
</ul>
</li>
<li><a class="reference internal" href="#custom-vectorization" id="id86">Custom vectorization</a><ul>
<li><a class="reference internal" href="#id31" id="id87">Introduction</a></li>
<li><a class="reference internal" href="#typed-list" id="id88">Typed list</a></li>
<li><a class="reference internal" href="#memory-aware-array" id="id89">Memory aware array</a></li>
<li><a class="reference internal" href="#id38" id="id90">Conclusion</a></li>
</ul>
</li>
<li><a class="reference internal" href="#beyond-numpy" id="id91">Beyond NumPy</a><ul>
<li><a class="reference internal" href="#back-to-python" id="id92">Back to Python</a></li>
<li><a class="reference internal" href="#numpy-co" id="id93">NumPy &amp; co</a></li>
<li><a class="reference internal" href="#scipy-co" id="id94">Scipy &amp; co</a></li>
<li><a class="reference internal" href="#id54" id="id95">Conclusion</a></li>
</ul>
</li>
<li><a class="reference internal" href="#id55" id="id96">Conclusion</a></li>
<li><a class="reference internal" href="#quick-references" id="id97">Quick References</a><ul>
<li><a class="reference internal" href="#id58" id="id98">Data type</a></li>
<li><a class="reference internal" href="#id59" id="id99">Creation</a></li>
<li><a class="reference internal" href="#id60" id="id100">Indexing</a></li>
<li><a class="reference internal" href="#reshaping" id="id101">Reshaping</a></li>
<li><a class="reference internal" href="#broadcasting" id="id102">Broadcasting</a></li>
</ul>
</li>
<li><a class="reference internal" href="#bibliography" id="id103">Bibliography</a><ul>
<li><a class="reference internal" href="#tutorials" id="id104">Tutorials</a></li>
<li><a class="reference internal" href="#articles" id="id105">Articles</a></li>
<li><a class="reference internal" href="#books" id="id106">Books</a></li>
</ul>
</li>
</ul>
</div>
<div class="line-block">
<div class="line"><br /></div>
<div class="line"><br /></div>
</div>
<p><strong>Disclaimer:</strong> All external pictures should have associated credits. If there
are missing credits, please tell me, I will correct it. Similarly, all excerpts
should be sourced (wikipedia mostly). If not, this is an error and I will
correct it as soon as you tell me.</p>
<p>The book is open-access (you're reading it) but <strong>if you insist on buying it</strong>,
my advice would be to read it first and then decide if you still want to buy it
(!?). If this is the case, you can do it via <a class="reference external" href="https://www.paypal.me/NicolasPRougier/">Paypal</a>, price is free (<a class="reference external" href="https://www.paypal.me/NicolasPRougier/5">5 euros</a>, <a class="reference external" href="https://www.paypal.me/NicolasPRougier/10">10 euros</a>, <a class="reference external" href="https://www.paypal.me/NicolasPRougier/25">25 euros</a>). You won't get anything extra but
it might help me with the writing of the upcoming <strong>Python and OpenGL for
Scientific Visualization</strong> (May 2018).</p>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<div class="section" id="preface">
<h1><a class="toc-backref" href="#table-of-contents">Preface</a></h1>
<div class="contents local topic" id="contents">
<p class="topic-title first"><strong>Contents</strong></p>
<ul class="simple">
<li><a class="reference internal" href="#about-the-author" id="id107">About the author</a></li>
<li><a class="reference internal" href="#about-this-book" id="id108">About this book</a><ul>
<li><a class="reference internal" href="#prerequisites" id="id109">Prerequisites</a></li>
<li><a class="reference internal" href="#conventions" id="id110">Conventions</a></li>
<li><a class="reference internal" href="#how-to-contribute" id="id111">How to contribute</a></li>
<li><a class="reference internal" href="#publishing" id="id112">Publishing</a></li>
</ul>
</li>
<li><a class="reference internal" href="#license" id="id113">License</a></li>
</ul>
</div>
<div class="section" id="about-the-author">
<h2><a class="toc-backref" href="#contents">About the author</a></h2>
<p><a class="reference external" href="http://www.labri.fr/perso/nrougier/">Nicolas P. Rougier</a> is a full-time research scientist at <a class="reference external" href="http://www.inria.fr/en">Inria</a> which is the
French national institute for research in computer science and control. This is
a public scientific and technological establishment (EPST) under the double
supervision of the Research &amp; Education Ministry, and the Ministry of Economy
Finance and Industry. Nicolas P. Rougier is working within the <a class="reference external" href="http://www.inria.fr/en/teams/mnemosyne">Mnemosyne</a>
project which lies at the frontier between integrative and computational
neuroscience in association with the <a class="reference external" href="http://www.imn-bordeaux.org/en/">Institute of Neurodegenerative
Diseases</a>, the Bordeaux laboratory for research in computer science
(<a class="reference external" href="https://www.labri.fr/">LaBRI</a>), the <a class="reference external" href="http://www.u-bordeaux.com/">University of Bordeaux</a> and the national center for scientific
research (<a class="reference external" href="http://www.cnrs.fr/index.php">CNRS</a>).</p>
<p>He has been using Python for more than 15 years and NumPy for more than 10
years for modeling in neuroscience, machine learning and for advanced
visualization (OpenGL). Nicolas P. Rougier is the author of several online
resources and tutorials (Matplotlib, NumPy, OpenGL) and he's teaching Python,
NumPy and scientific visualization at the University of Bordeaux and in various
conferences and schools worldwide (SciPy, EuroScipy, etc). He's also the author
of the popular article <a class="reference external" href="http://dx.doi.org/10.1371/journal.pcbi.1003833">Ten Simple Rules for Better Figures</a> and a popular
<a class="reference external" href="http://www.labri.fr/perso/nrougier/teaching/matplotlib/matplotlib.html">matplotlib tutorial</a>.</p>
</div>
<div class="section" id="about-this-book">
<h2><a class="toc-backref" href="#contents">About this book</a></h2>
<p>This book has been written in <a class="reference external" href="http://docutils.sourceforge.net/rst.html">restructured text</a> format and generated using the
<code>rst2html.py</code> command line available from the <a class="reference external" href="http://docutils.sourceforge.net/">docutils</a> python package.</p>
<p>If you want to rebuild the html output, from the top directory, type:</p>
<pre class="code literal-block">
$ rst2html.py --link-stylesheet --cloak-email-addresses \
              --toc-top-backlinks --stylesheet=book.css \
              --stylesheet-dirs=. book.rst book.html
</pre>
<p>The sources are available from <a class="reference external" href="https://github.com/rougier/from-python-to-numpy">https://github.com/rougier/from-python-to-numpy</a>.</p>
<div class="section" id="prerequisites">
<h3><a class="toc-backref" href="#contents">Prerequisites</a></h3>
<p>This is not a Python beginner guide and you should have an intermediate level in
Python and ideally a beginner level in NumPy. If this is not the case, have
a look at the <a class="reference internal" href="#bibliography">bibliography</a> for a curated list of resources.</p>
</div>
<div class="section" id="conventions">
<h3><a class="toc-backref" href="#contents">Conventions</a></h3>
<p>We will use usual naming conventions. If not stated explicitly, each script
should import NumPy, scipy and matplotlib as:</p>
<pre class="code python literal-block">
<span class="keyword namespace">import</span> <span class="name namespace">numpy</span> <span class="keyword namespace">as</span> <span class="name namespace">np</span>
<span class="keyword namespace">import</span> <span class="name namespace">scipy</span> <span class="keyword namespace">as</span> <span class="name namespace">sp</span>
<span class="keyword namespace">import</span> <span class="name namespace">matplotlib.pyplot</span> <span class="keyword namespace">as</span> <span class="name namespace">plt</span>
</pre>
<p>We'll use up-to-date versions (at the date of writing, i.e. January, 2017) of the
different packages:</p>
<table border="1" class="docutils">
<colgroup>
<col width="55%" />
<col width="45%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">Packages</th>
<th class="head">Version</th>
</tr>
</thead>
<tbody valign="top">
<tr><td>Python</td>
<td>3.6.0</td>
</tr>
<tr><td>NumPy</td>
<td>1.12.0</td>
</tr>
<tr><td>Scipy</td>
<td>0.18.1</td>
</tr>
<tr><td>Matplotlib</td>
<td>2.0.0</td>
</tr>
</tbody>
</table>
</div>
<div class="section" id="how-to-contribute">
<h3><a class="toc-backref" href="#contents">How to contribute</a></h3>
<p>If you want to contribute to this book, you can:</p>
<ul class="simple">
<li>Review chapters (please contact me)</li>
<li>Report issues (<a class="reference external" href="https://github.com/rougier/from-python-to-numpy/issues">https://github.com/rougier/from-python-to-numpy/issues</a>)</li>
<li>Suggest improvements (<a class="reference external" href="https://github.com/rougier/from-python-to-numpy/pulls">https://github.com/rougier/from-python-to-numpy/pulls</a>)</li>
<li>Correct English (<a class="reference external" href="https://github.com/rougier/from-python-to-numpy/issues">https://github.com/rougier/from-python-to-numpy/issues</a>)</li>
<li>Design a better and more responsive html template for the book.</li>
<li>Star the project (<a class="reference external" href="https://github.com/rougier/from-python-to-numpy">https://github.com/rougier/from-python-to-numpy</a>)</li>
</ul>
</div>
<div class="section" id="publishing">
<h3><a class="toc-backref" href="#contents">Publishing</a></h3>
<p>If you're an editor interested in publishing this book, you can <a class="reference external" href="mailto:Nicolas&#46;Rougier&#37;&#52;&#48;inria&#46;fr">contact me</a> if you agree to have this version and all
subsequent versions open access (i.e. online at <a class="reference external" href="http://www.labri.fr/perso/nrougier/from-python-to-numpy">this address</a>), you know how to
deal with <a class="reference external" href="http://docutils.sourceforge.net/rst.html">restructured text</a> (Word
is not an option), you provide a real added-value as well as supporting
services, and more importantly, you have a truly amazing latex book template
(and be warned that I'm a bit picky about typography &amp; design: <a class="reference external" href="https://www.edwardtufte.com/tufte/">Edward Tufte</a> is my hero). Still here?</p>
</div>
</div>
<div class="section" id="license">
<h2><a class="toc-backref" href="#contents">License</a></h2>
<p><strong>Book</strong></p>
<p>This work is licensed under a <a class="reference external" href="https://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-Non Commercial-Share
Alike 4.0 International License</a>. You are free to:</p>
<ul class="simple">
<li><strong>Share</strong> — copy and redistribute the material in any medium or format</li>
<li><strong>Adapt</strong> — remix, transform, and build upon the material</li>
</ul>
<p>The licensor cannot revoke these freedoms as long as you follow the license terms.</p>
<p><strong>Code</strong></p>
<p>The code is licensed under the <a class="reference external" href="LICENSE-code.txt">OSI-approved BSD 2-Clause License</a>.</p>
<!-- - - - Links - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
</div>
</div>
<div class="section" id="introduction">
<h1><a class="toc-backref" href="#table-of-contents">Introduction</a></h1>
<div class="contents local topic" id="id3">
<p class="topic-title first"><strong>Contents</strong></p>
<ul class="simple">
<li><a class="reference internal" href="#simple-example" id="id114">Simple example</a></li>
<li><a class="reference internal" href="#readability-vs-speed" id="id115">Readability vs speed</a></li>
</ul>
</div>
<div class="section" id="simple-example">
<h2><a class="toc-backref" href="#id3">Simple example</a></h2>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">You can execute any code below from the <a class="reference external" href="code">code</a> folder using the
regular python shell or from inside an IPython session or Jupyter notebook. In
such a case, you might want to use the magic command <code>%timeit</code> instead of the
<a class="reference external" href="code/tools.py">custom one</a> I wrote.</p>
</div>
<p>NumPy is all about vectorization. If you are familiar with Python, this is the
main difficulty you'll face because you'll need to change your way of thinking
and your new friends (among others) are named &quot;vectors&quot;, &quot;arrays&quot;, &quot;views&quot; or
&quot;ufuncs&quot;.</p>
<p>Let's take a very simple example, random walk. One possible object oriented
approach would be to define a <code>RandomWalker</code> class and write a walk
method that would return the current position after each (random) step. It's nice,
it's readable, but it is slow:</p>
<p><strong>Object oriented approach</strong></p>
<pre class="code python literal-block">
<span class="keyword">class</span> <span class="name class">RandomWalker</span><span class="punctuation">:</span>
    <span class="keyword">def</span> <span class="name function magic">__init__</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="punctuation">):</span>
        <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">position</span> <span class="operator">=</span> <span class="literal number integer">0</span>

    <span class="keyword">def</span> <span class="name function">walk</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="punctuation">,</span> <span class="name">n</span><span class="punctuation">):</span>
        <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">position</span> <span class="operator">=</span> <span class="literal number integer">0</span>
        <span class="keyword">for</span> <span class="name">i</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="name">n</span><span class="punctuation">):</span>
            <span class="keyword">yield</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">position</span>
            <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">position</span> <span class="operator">+=</span> <span class="literal number integer">2</span><span class="operator">*</span><span class="name">random</span><span class="operator">.</span><span class="name">randint</span><span class="punctuation">(</span><span class="literal number integer">0</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">)</span> <span class="operator">-</span> <span class="literal number integer">1</span>

<span class="name">walker</span> <span class="operator">=</span> <span class="name">RandomWalker</span><span class="punctuation">()</span>
<span class="name">walk</span> <span class="operator">=</span> <span class="punctuation">[</span><span class="name">position</span> <span class="keyword">for</span> <span class="name">position</span> <span class="operator word">in</span> <span class="name">walker</span><span class="operator">.</span><span class="name">walk</span><span class="punctuation">(</span><span class="literal number integer">1000</span><span class="punctuation">)]</span>
</pre>
<p>Benchmarking gives us:</p>
<pre class="code pycon literal-block">
<span class="keyword namespace"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="keyword namespace">from</span> <span class="name namespace">tools</span> <span class="keyword namespace">import</span> <span class="name">timeit</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="name">walker</span> <span class="operator">=</span> <span class="name">RandomWalker</span><span class="punctuation">()</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;[position for position in walker.walk(n=10000)]&quot;</span><span class="punctuation">,</span> <span class="name builtin">globals</span><span class="punctuation">())</span>
<span class="generic output">10 loops, best of 3: 15.7 msec per loop</span>
</pre>
<p><strong>Procedural approach</strong></p>
<p>For such a simple problem, we can probably save the class definition and
concentrate only on the walk method that computes successive positions after
each random step.</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function">random_walk</span><span class="punctuation">(</span><span class="name">n</span><span class="punctuation">):</span>
    <span class="name">position</span> <span class="operator">=</span> <span class="literal number integer">0</span>
    <span class="name">walk</span> <span class="operator">=</span> <span class="punctuation">[</span><span class="name">position</span><span class="punctuation">]</span>
    <span class="keyword">for</span> <span class="name">i</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="name">n</span><span class="punctuation">):</span>
        <span class="name">position</span> <span class="operator">+=</span> <span class="literal number integer">2</span><span class="operator">*</span><span class="name">random</span><span class="operator">.</span><span class="name">randint</span><span class="punctuation">(</span><span class="literal number integer">0</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">)</span><span class="operator">-</span><span class="literal number integer">1</span>
        <span class="name">walk</span><span class="operator">.</span><span class="name">append</span><span class="punctuation">(</span><span class="name">position</span><span class="punctuation">)</span>
    <span class="keyword">return</span> <span class="name">walk</span>

<span class="name">walk</span> <span class="operator">=</span> <span class="name">random_walk</span><span class="punctuation">(</span><span class="literal number integer">1000</span><span class="punctuation">)</span>
</pre>
<p>This new method saves some CPU cycles but not that much because this function
is pretty much the same as in the object-oriented approach and the few cycles
we saved probably come from the inner Python object-oriented machinery.</p>
<pre class="code pycon literal-block">
<span class="keyword namespace"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="keyword namespace">from</span> <span class="name namespace">tools</span> <span class="keyword namespace">import</span> <span class="name">timeit</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;random_walk(n=10000)&quot;</span><span class="punctuation">,</span> <span class="name builtin">globals</span><span class="punctuation">())</span>
<span class="generic output">10 loops, best of 3: 15.6 msec per loop</span>
</pre>
<p><strong>Vectorized approach</strong></p>
<p>But we can do better using the <a class="reference external" href="https://docs.python.org/3.6/library/itertools.html">itertools</a> Python module that
offers <em>a set of functions creating iterators for efficient looping</em>. If we
observe that a random walk is an accumulation of steps, we can rewrite the
function by first generating all the steps and accumulate them without any
loop:</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function">random_walk_faster</span><span class="punctuation">(</span><span class="name">n</span><span class="operator">=</span><span class="literal number integer">1000</span><span class="punctuation">):</span>
    <span class="keyword namespace">from</span> <span class="name namespace">itertools</span> <span class="keyword namespace">import</span> <span class="name">accumulate</span>
    <span class="comment single"># Only available from Python 3.6</span>
    <span class="name">steps</span> <span class="operator">=</span> <span class="name">random</span><span class="operator">.</span><span class="name">choices</span><span class="punctuation">([</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="operator">+</span><span class="literal number integer">1</span><span class="punctuation">],</span> <span class="name">k</span><span class="operator">=</span><span class="name">n</span><span class="punctuation">)</span>
    <span class="keyword">return</span> <span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">]</span><span class="operator">+</span><span class="name builtin">list</span><span class="punctuation">(</span><span class="name">accumulate</span><span class="punctuation">(</span><span class="name">steps</span><span class="punctuation">))</span>

 <span class="name">walk</span> <span class="operator">=</span> <span class="name">random_walk_faster</span><span class="punctuation">(</span><span class="literal number integer">1000</span><span class="punctuation">)</span>
</pre>
<p>In fact, we've just <em>vectorized</em> our function. Instead of looping for picking
sequential steps and add them to the current position, we first generated all the
steps at once and used the <a class="reference external" href="https://docs.python.org/3.6/library/itertools.html#itertools.accumulate">accumulate</a>
function to compute all the positions. We got rid of the loop and this makes
things faster:</p>
<pre class="code pycon literal-block">
<span class="keyword namespace"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="keyword namespace">from</span> <span class="name namespace">tools</span> <span class="keyword namespace">import</span> <span class="name">timeit</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;random_walk_faster(n=10000)&quot;</span><span class="punctuation">,</span> <span class="name builtin">globals</span><span class="punctuation">())</span>
<span class="generic output">10 loops, best of 3: 2.21 msec per loop</span>
</pre>
<p>We gained 85% of computation-time compared to the previous version, not so
bad. But the advantage of this new version is that it makes NumPy vectorization
super simple. We just have to translate itertools call into NumPy ones.</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function">random_walk_fastest</span><span class="punctuation">(</span><span class="name">n</span><span class="operator">=</span><span class="literal number integer">1000</span><span class="punctuation">):</span>
    <span class="comment single"># No 's' in NumPy choice (Python offers choice &amp; choices)</span>
    <span class="name">steps</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">random</span><span class="operator">.</span><span class="name">choice</span><span class="punctuation">([</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="operator">+</span><span class="literal number integer">1</span><span class="punctuation">],</span> <span class="name">n</span><span class="punctuation">)</span>
    <span class="keyword">return</span> <span class="name">np</span><span class="operator">.</span><span class="name">cumsum</span><span class="punctuation">(</span><span class="name">steps</span><span class="punctuation">)</span>

<span class="name">walk</span> <span class="operator">=</span> <span class="name">random_walk_fastest</span><span class="punctuation">(</span><span class="literal number integer">1000</span><span class="punctuation">)</span>
</pre>
<p>Not too difficult, but we gained a factor 500x using NumPy:</p>
<pre class="code pycon literal-block">
<span class="keyword namespace"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="keyword namespace">from</span> <span class="name namespace">tools</span> <span class="keyword namespace">import</span> <span class="name">timeit</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;random_walk_fastest(n=10000)&quot;</span><span class="punctuation">,</span> <span class="name builtin">globals</span><span class="punctuation">())</span>
<span class="generic output">1000 loops, best of 3: 14 usec per loop</span>
</pre>
<p>This book is about vectorization, be it at the code or problem level. We'll
see this difference is important before looking at custom vectorization.</p>
</div>
<div class="section" id="readability-vs-speed">
<h2><a class="toc-backref" href="#id3">Readability vs speed</a></h2>
<p>Before heading to the next chapter, I would like to warn you about a potential
problem you may encounter once you'll have become familiar with NumPy. It is a
very powerful library and you can make wonders with it but, most of the time,
this comes at the price of readability. If you don't comment your code at the
time of writing, you won't be able to tell what a function is doing after a few
weeks (or possibly days). For example, can you tell what the two functions
below are doing? Probably you can tell for the first one, but unlikely for the
second (or your name is <a class="reference external" href="http://stackoverflow.com/questions/7100242/python-numpy-first-occurrence-of-subarray">Jaime Fernández del Río</a>
and you don't need to read this book).</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function">function_1</span><span class="punctuation">(</span><span class="name">seq</span><span class="punctuation">,</span> <span class="name">sub</span><span class="punctuation">):</span>
    <span class="keyword">return</span> <span class="punctuation">[</span><span class="name">i</span> <span class="keyword">for</span> <span class="name">i</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="name builtin">len</span><span class="punctuation">(</span><span class="name">seq</span><span class="punctuation">)</span> <span class="operator">-</span> <span class="name builtin">len</span><span class="punctuation">(</span><span class="name">sub</span><span class="punctuation">)</span> <span class="operator">+</span><span class="literal number integer">1</span><span class="punctuation">)</span> <span class="keyword">if</span> <span class="name">seq</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">:</span><span class="name">i</span><span class="operator">+</span><span class="name builtin">len</span><span class="punctuation">(</span><span class="name">sub</span><span class="punctuation">)]</span> <span class="operator">==</span> <span class="name">sub</span><span class="punctuation">]</span>

<span class="keyword">def</span> <span class="name function">function_2</span><span class="punctuation">(</span><span class="name">seq</span><span class="punctuation">,</span> <span class="name">sub</span><span class="punctuation">):</span>
    <span class="name">target</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">dot</span><span class="punctuation">(</span><span class="name">sub</span><span class="punctuation">,</span> <span class="name">sub</span><span class="punctuation">)</span>
    <span class="name">candidates</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">where</span><span class="punctuation">(</span><span class="name">np</span><span class="operator">.</span><span class="name">correlate</span><span class="punctuation">(</span><span class="name">seq</span><span class="punctuation">,</span> <span class="name">sub</span><span class="punctuation">,</span> <span class="name">mode</span><span class="operator">=</span><span class="literal string single">'valid'</span><span class="punctuation">)</span> <span class="operator">==</span> <span class="name">target</span><span class="punctuation">)[</span><span class="literal number integer">0</span><span class="punctuation">]</span>
    <span class="name">check</span> <span class="operator">=</span> <span class="name">candidates</span><span class="punctuation">[:,</span> <span class="name">np</span><span class="operator">.</span><span class="name">newaxis</span><span class="punctuation">]</span> <span class="operator">+</span> <span class="name">np</span><span class="operator">.</span><span class="name">arange</span><span class="punctuation">(</span><span class="name builtin">len</span><span class="punctuation">(</span><span class="name">sub</span><span class="punctuation">))</span>
    <span class="name">mask</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">all</span><span class="punctuation">((</span><span class="name">np</span><span class="operator">.</span><span class="name">take</span><span class="punctuation">(</span><span class="name">seq</span><span class="punctuation">,</span> <span class="name">check</span><span class="punctuation">)</span> <span class="operator">==</span> <span class="name">sub</span><span class="punctuation">),</span> <span class="name">axis</span><span class="operator">=-</span><span class="literal number integer">1</span><span class="punctuation">)</span>
    <span class="keyword">return</span> <span class="name">candidates</span><span class="punctuation">[</span><span class="name">mask</span><span class="punctuation">]</span>
</pre>
<p>As you may have guessed, the second function is the
vectorized-optimized-faster-NumPy version of the first function. It is 10 times
faster than the pure Python version, but it is hardly readable.</p>
</div>
</div>
<div class="section" id="anatomy-of-an-array">
<h1><a class="toc-backref" href="#table-of-contents">Anatomy of an array</a></h1>
<div class="contents local topic" id="id4">
<p class="topic-title first"><strong>Contents</strong></p>
<ul class="simple">
<li><a class="reference internal" href="#id5" id="id116">Introduction</a></li>
<li><a class="reference internal" href="#memory-layout" id="id117">Memory layout</a></li>
<li><a class="reference internal" href="#views-and-copies" id="id118">Views and copies</a><ul>
<li><a class="reference internal" href="#direct-and-indirect-access" id="id119">Direct and indirect access</a></li>
<li><a class="reference internal" href="#temporary-copy" id="id120">Temporary copy</a></li>
</ul>
</li>
<li><a class="reference internal" href="#conclusion" id="id121">Conclusion</a></li>
</ul>
</div>
<div class="section" id="id5">
<h2><a class="toc-backref" href="#id4">Introduction</a></h2>
<p>As explained in the <a class="reference internal" href="#preface">Preface</a>, you should have a basic experience with NumPy to
read this book. If this is not the case, you'd better start with a beginner
tutorial before coming back here. Consequently I'll only give here a quick
reminder on the basic anatomy of NumPy arrays, especially regarding the memory
layout, view, copy and the data type. They are critical notions to
understand if you want your computation to benefit from NumPy philosophy.</p>
<p>Let's consider a simple example where we want to clear all the values from an
array which has the dtype <code>np.float32</code>. How does one write it to maximize speed? The
below syntax is rather obvious (at least for those familiar with NumPy) but the
above question asks to find the fastest operation.</p>
<pre class="code python literal-block">
<span class="operator">&gt;&gt;&gt;</span> <span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">ones</span><span class="punctuation">(</span><span class="literal number integer">4</span><span class="operator">*</span><span class="literal number integer">1000000</span><span class="punctuation">,</span> <span class="name">np</span><span class="operator">.</span><span class="name">float32</span><span class="punctuation">)</span>
<span class="operator">&gt;&gt;&gt;</span> <span class="name">Z</span><span class="punctuation">[</span><span class="operator">...</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="literal number integer">0</span>
</pre>
<p>If you look more closely at both the dtype and the size of the array, you can
observe that this array can be casted (i.e. viewed) into many other
&quot;compatible&quot; data types. By compatible, I mean that <code>Z.size * Z.itemsize</code> can
be divided by the new dtype itemsize.</p>
<pre class="code python literal-block">
<span class="operator">&gt;&gt;&gt;</span> <span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;Z.view(np.float16)[...] = 0&quot;</span><span class="punctuation">,</span> <span class="name builtin">globals</span><span class="punctuation">())</span>
<span class="literal number integer">100</span> <span class="name">loops</span><span class="punctuation">,</span> <span class="name">best</span> <span class="name">of</span> <span class="literal number integer">3</span><span class="punctuation">:</span> <span class="literal number float">2.72</span> <span class="name">msec</span> <span class="name">per</span> <span class="name">loop</span>
<span class="operator">&gt;&gt;&gt;</span> <span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;Z.view(np.int16)[...] = 0&quot;</span><span class="punctuation">,</span> <span class="name builtin">globals</span><span class="punctuation">())</span>
<span class="literal number integer">100</span> <span class="name">loops</span><span class="punctuation">,</span> <span class="name">best</span> <span class="name">of</span> <span class="literal number integer">3</span><span class="punctuation">:</span> <span class="literal number float">2.77</span> <span class="name">msec</span> <span class="name">per</span> <span class="name">loop</span>
<span class="operator">&gt;&gt;&gt;</span> <span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;Z.view(np.int32)[...] = 0&quot;</span><span class="punctuation">,</span> <span class="name builtin">globals</span><span class="punctuation">())</span>
<span class="literal number integer">100</span> <span class="name">loops</span><span class="punctuation">,</span> <span class="name">best</span> <span class="name">of</span> <span class="literal number integer">3</span><span class="punctuation">:</span> <span class="literal number float">1.29</span> <span class="name">msec</span> <span class="name">per</span> <span class="name">loop</span>
<span class="operator">&gt;&gt;&gt;</span> <span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;Z.view(np.float32)[...] = 0&quot;</span><span class="punctuation">,</span> <span class="name builtin">globals</span><span class="punctuation">())</span>
<span class="literal number integer">100</span> <span class="name">loops</span><span class="punctuation">,</span> <span class="name">best</span> <span class="name">of</span> <span class="literal number integer">3</span><span class="punctuation">:</span> <span class="literal number float">1.33</span> <span class="name">msec</span> <span class="name">per</span> <span class="name">loop</span>
<span class="operator">&gt;&gt;&gt;</span> <span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;Z.view(np.int64)[...] = 0&quot;</span><span class="punctuation">,</span> <span class="name builtin">globals</span><span class="punctuation">())</span>
<span class="literal number integer">100</span> <span class="name">loops</span><span class="punctuation">,</span> <span class="name">best</span> <span class="name">of</span> <span class="literal number integer">3</span><span class="punctuation">:</span> <span class="literal number integer">874</span> <span class="name">usec</span> <span class="name">per</span> <span class="name">loop</span>
<span class="operator">&gt;&gt;&gt;</span> <span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;Z.view(np.float64)[...] = 0&quot;</span><span class="punctuation">,</span> <span class="name builtin">globals</span><span class="punctuation">())</span>
<span class="literal number integer">100</span> <span class="name">loops</span><span class="punctuation">,</span> <span class="name">best</span> <span class="name">of</span> <span class="literal number integer">3</span><span class="punctuation">:</span> <span class="literal number integer">865</span> <span class="name">usec</span> <span class="name">per</span> <span class="name">loop</span>
<span class="operator">&gt;&gt;&gt;</span> <span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;Z.view(np.complex128)[...] = 0&quot;</span><span class="punctuation">,</span> <span class="name builtin">globals</span><span class="punctuation">())</span>
<span class="literal number integer">100</span> <span class="name">loops</span><span class="punctuation">,</span> <span class="name">best</span> <span class="name">of</span> <span class="literal number integer">3</span><span class="punctuation">:</span> <span class="literal number integer">841</span> <span class="name">usec</span> <span class="name">per</span> <span class="name">loop</span>
<span class="operator">&gt;&gt;&gt;</span> <span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;Z.view(np.int8)[...] = 0&quot;</span><span class="punctuation">,</span> <span class="name builtin">globals</span><span class="punctuation">())</span>
<span class="literal number integer">100</span> <span class="name">loops</span><span class="punctuation">,</span> <span class="name">best</span> <span class="name">of</span> <span class="literal number integer">3</span><span class="punctuation">:</span> <span class="literal number integer">630</span> <span class="name">usec</span> <span class="name">per</span> <span class="name">loop</span>
</pre>
<p>Interestingly enough, the obvious way of clearing all the values is not the
fastest. By casting the array into a larger data type such as <code>np.float64</code>, we
gained a 25% speed factor. But, by viewing the array as a byte array
(<code>np.int8</code>), we gained a 50% factor. The reason for such speedup are to be
found in the internal NumPy machinery and the compiler optimization. This
simple example illustrates the philosophy of NumPy as we'll see in the next
section below.</p>
</div>
<div class="section" id="memory-layout">
<h2><a class="toc-backref" href="#id4">Memory layout</a></h2>
<p>The <a class="reference external" href="https://docs.scipy.org/doc/numpy/reference/arrays.ndarray.html">NumPy documentation</a> defines the
ndarray class very clearly:</p>
<blockquote>
<em>An instance of class ndarray consists of a contiguous one-dimensional segment
of computer memory (owned by the array, or by some other object), combined
with an indexing scheme that maps N integers into the location of an item in
the block.</em></blockquote>
<p>Said differently, an array is mostly a contiguous block of memory whose parts
can be accessed using an indexing scheme. Such indexing scheme is in turn
defined by a <a class="reference external" href="https://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.shape.html#numpy.ndarray.shape">shape</a>
and a <a class="reference external" href="https://docs.scipy.org/doc/numpy/reference/arrays.dtypes.html">data type</a> and this is
precisely what is needed when you define a new array:</p>
<pre class="code python literal-block">
<span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">arange</span><span class="punctuation">(</span><span class="literal number integer">9</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="literal number integer">3</span><span class="punctuation">,</span><span class="literal number integer">3</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">astype</span><span class="punctuation">(</span><span class="name">np</span><span class="operator">.</span><span class="name">int16</span><span class="punctuation">)</span>
</pre>
<p>Here, we know that Z itemsize is 2 bytes (<code>int16</code>), the shape is (3,3) and
the number of dimensions is 2 (<code>len(Z.shape)</code>).</p>
<pre class="code pycon literal-block">
<span class="keyword"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="keyword">print</span><span class="punctuation">(</span><span class="name">Z</span><span class="operator">.</span><span class="name">itemsize</span><span class="punctuation">)</span>
<span class="generic output">2
</span><span class="keyword"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="keyword">print</span><span class="punctuation">(</span><span class="name">Z</span><span class="operator">.</span><span class="name">shape</span><span class="punctuation">)</span>
<span class="generic output">(3, 3)
</span><span class="keyword"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="keyword">print</span><span class="punctuation">(</span><span class="name">Z</span><span class="operator">.</span><span class="name">ndim</span><span class="punctuation">)</span>
<span class="generic output">2</span>
</pre>
<p>Furthermore and because Z is not a view, we can deduce the
<a class="reference external" href="https://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.strides.html#numpy.ndarray.strides">strides</a> of the array that define the number of bytes to step in each dimension when traversing the array.</p>
<pre class="code pycon literal-block">
<span class="name"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="name">strides</span> <span class="operator">=</span> <span class="name">Z</span><span class="operator">.</span><span class="name">shape</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">]</span><span class="operator">*</span><span class="name">Z</span><span class="operator">.</span><span class="name">itemsize</span><span class="punctuation">,</span> <span class="name">Z</span><span class="operator">.</span><span class="name">itemsize</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="keyword">print</span><span class="punctuation">(</span><span class="name">strides</span><span class="punctuation">)</span>
<span class="generic output">(6, 2)
</span><span class="keyword"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="keyword">print</span><span class="punctuation">(</span><span class="name">Z</span><span class="operator">.</span><span class="name">strides</span><span class="punctuation">)</span>
<span class="generic output">(6, 2)</span>
</pre>
<p>With all these information, we know how to access a specific item (designed by
an index tuple) and more precisely, how to compute the start and end offsets:</p>
<pre class="code python literal-block">
<span class="name">offset_start</span> <span class="operator">=</span> <span class="literal number integer">0</span>
<span class="keyword">for</span> <span class="name">i</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="name">Z</span><span class="operator">.</span><span class="name">ndim</span><span class="punctuation">):</span>
    <span class="name">offset_start</span> <span class="operator">+=</span> <span class="name">Z</span><span class="operator">.</span><span class="name">strides</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">]</span> <span class="operator">*</span> <span class="name">index</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">]</span>
<span class="name">offset_end</span> <span class="operator">=</span> <span class="name">offset_start</span> <span class="operator">+</span> <span class="name">Z</span><span class="operator">.</span><span class="name">itemsize</span>
</pre>
<p>Let's see if this is correct using the <a class="reference external" href="https://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.tobytes.html">tobytes</a>
conversion method:</p>
<pre class="code python literal-block">
<span class="operator">&gt;&gt;&gt;</span> <span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">arange</span><span class="punctuation">(</span><span class="literal number integer">9</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="literal number integer">3</span><span class="punctuation">,</span> <span class="literal number integer">3</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">astype</span><span class="punctuation">(</span><span class="name">np</span><span class="operator">.</span><span class="name">int16</span><span class="punctuation">)</span>
<span class="operator">&gt;&gt;&gt;</span> <span class="name">index</span> <span class="operator">=</span> <span class="literal number integer">1</span><span class="punctuation">,</span> <span class="literal number integer">1</span>
<span class="operator">&gt;&gt;&gt;</span> <span class="keyword">print</span><span class="punctuation">(</span><span class="name">Z</span><span class="punctuation">[</span><span class="name">index</span><span class="punctuation">]</span><span class="operator">.</span><span class="name">tobytes</span><span class="punctuation">())</span>
<span class="literal string affix">b</span><span class="literal string single">'</span><span class="literal string escape">\x04\x00</span><span class="literal string single">'</span>
<span class="operator">&gt;&gt;&gt;</span> <span class="name">offset_start</span> <span class="operator">=</span> <span class="literal number integer">0</span>
<span class="operator">&gt;&gt;&gt;</span> <span class="keyword">for</span> <span class="name">i</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="name">Z</span><span class="operator">.</span><span class="name">ndim</span><span class="punctuation">):</span>
<span class="operator">...</span>     <span class="name">offset_start</span> <span class="operator">+=</span> <span class="name">Z</span><span class="operator">.</span><span class="name">strides</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">]</span> <span class="operator">*</span> <span class="name">index</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">]</span>
<span class="operator">&gt;&gt;&gt;</span> <span class="name">offset_end</span> <span class="operator">=</span> <span class="name">offset_start</span> <span class="operator">+</span> <span class="name">Z</span><span class="operator">.</span><span class="name">itemsize</span>
<span class="operator">&gt;&gt;&gt;</span> <span class="keyword">print</span><span class="punctuation">(</span><span class="name">Z</span><span class="operator">.</span><span class="name">tobytes</span><span class="punctuation">()[</span><span class="name">offset_start</span><span class="punctuation">:</span><span class="name">offset_end</span><span class="punctuation">]</span>
<span class="literal string affix">b</span><span class="literal string single">'</span><span class="literal string escape">\x04\x00</span><span class="literal string single">'</span>
</pre>
<p>This array can be actually considered from different perspectives (i.e. layouts):</p>
<p><strong>Item layout</strong></p>
<pre class="code output literal-block">
               shape[1]
                 (=3)
            ┌───────────┐

         ┌  ┌───┬───┬───┐  ┐
         │  │ 0 │ 1 │ 2 │  │
         │  ├───┼───┼───┤  │
shape[0] │  │ 3 │ 4 │ 5 │  │ len(Z)
 (=3)    │  ├───┼───┼───┤  │  (=3)
         │  │ 6 │ 7 │ 8 │  │
         └  └───┴───┴───┘  ┘
</pre>
<p><strong>Flattened item layout</strong></p>
<pre class="code output literal-block">
┌───┬───┬───┬───┬───┬───┬───┬───┬───┐
│ 0 │ 1 │ 2 │ 3 │ 4 │ 5 │ 6 │ 7 │ 8 │
└───┴───┴───┴───┴───┴───┴───┴───┴───┘

└───────────────────────────────────┘
               Z.size
                (=9)
</pre>
<p><strong>Memory layout (C order, big endian)</strong></p>
<pre class="code output literal-block">
                         strides[1]
                           (=2)
                  ┌─────────────────────┐

          ┌       ┌──────────┬──────────┐ ┐
          │ p+00: │ 00000000 │ 00000000 │ │
          │       ├──────────┼──────────┤ │
          │ p+02: │ 00000000 │ 00000001 │ │ strides[0]
          │       ├──────────┼──────────┤ │   (=2x3)
          │ p+04  │ 00000000 │ 00000010 │ │
          │       ├──────────┼──────────┤ ┘
          │ p+06  │ 00000000 │ 00000011 │
          │       ├──────────┼──────────┤
Z.nbytes  │ p+08: │ 00000000 │ 00000100 │
(=3x3x2)  │       ├──────────┼──────────┤
          │ p+10: │ 00000000 │ 00000101 │
          │       ├──────────┼──────────┤
          │ p+12: │ 00000000 │ 00000110 │
          │       ├──────────┼──────────┤
          │ p+14: │ 00000000 │ 00000111 │
          │       ├──────────┼──────────┤
          │ p+16: │ 00000000 │ 00001000 │
          └       └──────────┴──────────┘

                  └─────────────────────┘
                        Z.itemsize
                     Z.dtype.itemsize
                           (=2)
</pre>
<p>If we now take a slice of <code>Z</code>, the result is a view of the base array <code>Z</code>:</p>
<pre class="code python literal-block">
<span class="name">V</span> <span class="operator">=</span> <span class="name">Z</span><span class="punctuation">[::</span><span class="literal number integer">2</span><span class="punctuation">,::</span><span class="literal number integer">2</span><span class="punctuation">]</span>
</pre>
<p>Such view is specified using a shape, a dtype <strong>and</strong> strides because strides
cannot be deduced anymore from the dtype and shape only:</p>
<p><strong>Item layout</strong></p>
<pre class="code output literal-block">
               shape[1]
                 (=2)
            ┌───────────┐

         ┌  ┌───┬╌╌╌┬───┐  ┐
         │  │ 0 │   │ 2 │  │            ┌───┬───┐
         │  ├───┼╌╌╌┼───┤  │            │ 0 │ 2 │
shape[0] │  ╎   ╎   ╎   ╎  │ len(Z)  →  ├───┼───┤
 (=2)    │  ├───┼╌╌╌┼───┤  │  (=2)      │ 6 │ 8 │
         │  │ 6 │   │ 8 │  │            └───┴───┘
         └  └───┴╌╌╌┴───┘  ┘
</pre>
<p><strong>Flattened item layout</strong></p>
<pre class="code output literal-block">
┌───┬╌╌╌┬───┬╌╌╌┬╌╌╌┬╌╌╌┬───┬╌╌╌┬───┐       ┌───┬───┬───┬───┐
│ 0 │   │ 2 │   ╎   ╎   │ 6 │   │ 8 │   →   │ 0 │ 2 │ 6 │ 8 │
└───┴╌╌╌┴───┴╌╌╌┴╌╌╌┴╌╌╌┴───┴╌╌╌┴───┘       └───┴───┴───┴───┘
└─┬─┘   └─┬─┘           └─┬─┘   └─┬─┘
  └───┬───┘               └───┬───┘
      └───────────┬───────────┘
               Z.size
                (=4)
</pre>
<p><strong>Memory layout (C order, big endian)</strong></p>
<pre class="code output literal-block">
              ┌        ┌──────────┬──────────┐ ┐             ┐
            ┌─┤  p+00: │ 00000000 │ 00000000 │ │             │
            │ └        ├──────────┼──────────┤ │ strides[1]  │
          ┌─┤    p+02: │          │          │ │   (=4)      │
          │ │ ┌        ├──────────┼──────────┤ ┘             │
          │ └─┤  p+04  │ 00000000 │ 00000010 │               │
          │   └        ├──────────┼──────────┤               │ strides[0]
          │      p+06: │          │          │               │   (=12)
          │            ├──────────┼──────────┤               │
Z.nbytes ─┤      p+08: │          │          │               │
  (=8)    │            ├──────────┼──────────┤               │
          │      p+10: │          │          │               │
          │   ┌        ├──────────┼──────────┤               ┘
          │ ┌─┤  p+12: │ 00000000 │ 00000110 │
          │ │ └        ├──────────┼──────────┤
          └─┤    p+14: │          │          │
            │ ┌        ├──────────┼──────────┤
            └─┤  p+16: │ 00000000 │ 00001000 │
              └        └──────────┴──────────┘

                       └─────────────────────┘
                             Z.itemsize
                          Z.dtype.itemsize
                                (=2)
</pre>
</div>
<div class="section" id="views-and-copies">
<h2><a class="toc-backref" href="#id4">Views and copies</a></h2>
<p>Views and copies are important concepts for the optimization of your numerical
computations. Even if we've already manipulated them in the previous section,
the whole story is a bit more complex.</p>
<div class="section" id="direct-and-indirect-access">
<h3><a class="toc-backref" href="#id4">Direct and indirect access</a></h3>
<p>First, we have to distinguish between <a class="reference external" href="https://docs.scipy.org/doc/numpy/user/basics.indexing.html#">indexing</a> and <a class="reference external" href="https://docs.scipy.org/doc/numpy/reference/arrays.indexing.html#advanced-indexing">fancy
indexing</a>. The first will always return a view while the second will return a
copy. This difference is important because in the first case, modifying the view
modifies the base array while this is not true in the second case:</p>
<pre class="code pycon literal-block">
<span class="name"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">zeros</span><span class="punctuation">(</span><span class="literal number integer">9</span><span class="punctuation">)</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="name">Z_view</span> <span class="operator">=</span> <span class="name">Z</span><span class="punctuation">[:</span><span class="literal number integer">3</span><span class="punctuation">]</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="name">Z_view</span><span class="punctuation">[</span><span class="operator">...</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="literal number integer">1</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="keyword">print</span><span class="punctuation">(</span><span class="name">Z</span><span class="punctuation">)</span>
<span class="generic output">[ 1.  1.  1.  0.  0.  0.  0.  0.  0.]
</span><span class="name"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">zeros</span><span class="punctuation">(</span><span class="literal number integer">9</span><span class="punctuation">)</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="name">Z_copy</span> <span class="operator">=</span> <span class="name">Z</span><span class="punctuation">[[</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">2</span><span class="punctuation">]]</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="name">Z_copy</span><span class="punctuation">[</span><span class="operator">...</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="literal number integer">1</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="keyword">print</span><span class="punctuation">(</span><span class="name">Z</span><span class="punctuation">)</span>
<span class="generic output">[ 0.  0.  0.  0.  0.  0.  0.  0.  0.]</span>
</pre>
<p>Thus, if you need fancy indexing, it's better to keep a copy of your fancy index
(especially if it was complex to compute it) and to work with it:</p>
<pre class="code pycon literal-block">
<span class="name"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">zeros</span><span class="punctuation">(</span><span class="literal number integer">9</span><span class="punctuation">)</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="name">index</span> <span class="operator">=</span> <span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">2</span><span class="punctuation">]</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="name">Z</span><span class="punctuation">[</span><span class="name">index</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="literal number integer">1</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="keyword">print</span><span class="punctuation">(</span><span class="name">Z</span><span class="punctuation">)</span>
<span class="generic output">[ 1.  1.  1.  0.  0.  0.  0.  0.  0.]</span>
</pre>
<p>If you are unsure if the result of your indexing is a view or a copy, you can
check what is the <code>base</code> of your result. If it is <code>None</code>, then you result is a
copy:</p>
<pre class="code pycon literal-block">
<span class="name"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">random</span><span class="operator">.</span><span class="name">uniform</span><span class="punctuation">(</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">,(</span><span class="literal number integer">5</span><span class="punctuation">,</span><span class="literal number integer">5</span><span class="punctuation">))</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="name">Z1</span> <span class="operator">=</span> <span class="name">Z</span><span class="punctuation">[:</span><span class="literal number integer">3</span><span class="punctuation">,:]</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="name">Z2</span> <span class="operator">=</span> <span class="name">Z</span><span class="punctuation">[[</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">2</span><span class="punctuation">],</span> <span class="punctuation">:]</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="keyword">print</span><span class="punctuation">(</span><span class="name">np</span><span class="operator">.</span><span class="name">allclose</span><span class="punctuation">(</span><span class="name">Z1</span><span class="punctuation">,</span><span class="name">Z2</span><span class="punctuation">))</span>
<span class="generic output">True
</span><span class="keyword"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="keyword">print</span><span class="punctuation">(</span><span class="name">Z1</span><span class="operator">.</span><span class="name">base</span> <span class="operator word">is</span> <span class="name">Z</span><span class="punctuation">)</span>
<span class="generic output">True
</span><span class="keyword"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="keyword">print</span><span class="punctuation">(</span><span class="name">Z2</span><span class="operator">.</span><span class="name">base</span> <span class="operator word">is</span> <span class="name">Z</span><span class="punctuation">)</span>
<span class="generic output">False
</span><span class="keyword"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="keyword">print</span><span class="punctuation">(</span><span class="name">Z2</span><span class="operator">.</span><span class="name">base</span> <span class="operator word">is</span> <span class="name builtin pseudo">None</span><span class="punctuation">)</span>
<span class="generic output">True</span>
</pre>
<p>Note that some NumPy functions return a view when possible (e.g. <a class="reference external" href="https://docs.scipy.org/doc/numpy/reference/generated/numpy.ravel.html">ravel</a>)
while some others always return a copy (e.g. <a class="reference external" href="https://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.flatten.html#numpy.ndarray.flatten">flatten</a>):</p>
<pre class="code pycon literal-block">
<span class="name"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">zeros</span><span class="punctuation">((</span><span class="literal number integer">5</span><span class="punctuation">,</span><span class="literal number integer">5</span><span class="punctuation">))</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="name">Z</span><span class="operator">.</span><span class="name">ravel</span><span class="punctuation">()</span><span class="operator">.</span><span class="name">base</span> <span class="operator word">is</span> <span class="name">Z</span>
<span class="generic output">True
</span><span class="name"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="name">Z</span><span class="punctuation">[::</span><span class="literal number integer">2</span><span class="punctuation">,::</span><span class="literal number integer">2</span><span class="punctuation">]</span><span class="operator">.</span><span class="name">ravel</span><span class="punctuation">()</span><span class="operator">.</span><span class="name">base</span> <span class="operator word">is</span> <span class="name">Z</span>
<span class="generic output">False
</span><span class="name"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="name">Z</span><span class="operator">.</span><span class="name">flatten</span><span class="punctuation">()</span><span class="operator">.</span><span class="name">base</span> <span class="operator word">is</span> <span class="name">Z</span>
<span class="generic output">False</span>
</pre>
</div>
<div class="section" id="temporary-copy">
<h3><a class="toc-backref" href="#id4">Temporary copy</a></h3>
<p>Copies can be made explicitly like in the previous section, but the most
general case is the implicit creation of intermediate copies. This is the case
when you are doing some arithmetic with arrays:</p>
<pre class="code pycon literal-block">
<span class="name"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="name">X</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">ones</span><span class="punctuation">(</span><span class="literal number integer">10</span><span class="punctuation">,</span> <span class="name">dtype</span><span class="operator">=</span><span class="name">np</span><span class="operator">.</span><span class="name">int</span><span class="punctuation">)</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="name">Y</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">ones</span><span class="punctuation">(</span><span class="literal number integer">10</span><span class="punctuation">,</span> <span class="name">dtype</span><span class="operator">=</span><span class="name">np</span><span class="operator">.</span><span class="name">int</span><span class="punctuation">)</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="name">A</span> <span class="operator">=</span> <span class="literal number integer">2</span><span class="operator">*</span><span class="name">X</span> <span class="operator">+</span> <span class="literal number integer">2</span><span class="operator">*</span><span class="name">Y</span>
</pre>
<p>In the example above, three intermediate arrays have been created. One for
holding the result of <code>2*X</code>, one for holding the result of <code>2*Y</code> and the last
one for holding the result of <code>2*X+2*Y</code>. In this specific case, the arrays are
small enough and this does not really make a difference. However, if your
arrays are big, then you have to be careful with such expressions and wonder if
you can do it differently. For example, if only the final result matters and
you don't need <code>X</code> nor <code>Y</code> afterwards, an alternate solution would be:</p>
<pre class="code pycon literal-block">
<span class="name"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="name">X</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">ones</span><span class="punctuation">(</span><span class="literal number integer">10</span><span class="punctuation">,</span> <span class="name">dtype</span><span class="operator">=</span><span class="name">np</span><span class="operator">.</span><span class="name">int</span><span class="punctuation">)</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="name">Y</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">ones</span><span class="punctuation">(</span><span class="literal number integer">10</span><span class="punctuation">,</span> <span class="name">dtype</span><span class="operator">=</span><span class="name">np</span><span class="operator">.</span><span class="name">int</span><span class="punctuation">)</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="name">np</span><span class="operator">.</span><span class="name">multiply</span><span class="punctuation">(</span><span class="name">X</span><span class="punctuation">,</span> <span class="literal number integer">2</span><span class="punctuation">,</span> <span class="name">out</span><span class="operator">=</span><span class="name">X</span><span class="punctuation">)</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="name">np</span><span class="operator">.</span><span class="name">multiply</span><span class="punctuation">(</span><span class="name">Y</span><span class="punctuation">,</span> <span class="literal number integer">2</span><span class="punctuation">,</span> <span class="name">out</span><span class="operator">=</span><span class="name">Y</span><span class="punctuation">)</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="name">np</span><span class="operator">.</span><span class="name">add</span><span class="punctuation">(</span><span class="name">X</span><span class="punctuation">,</span> <span class="name">Y</span><span class="punctuation">,</span> <span class="name">out</span><span class="operator">=</span><span class="name">X</span><span class="punctuation">)</span>
</pre>
<p>Using this alternate solution, no temporary array has been created. Problem is
that there are many other cases where such copies needs to be created and this
impact the performance like demonstrated on the example below:</p>
<pre class="code pycon literal-block">
<span class="name"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="name">X</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">ones</span><span class="punctuation">(</span><span class="literal number integer">1000000000</span><span class="punctuation">,</span> <span class="name">dtype</span><span class="operator">=</span><span class="name">np</span><span class="operator">.</span><span class="name">int</span><span class="punctuation">)</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="name">Y</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">ones</span><span class="punctuation">(</span><span class="literal number integer">1000000000</span><span class="punctuation">,</span> <span class="name">dtype</span><span class="operator">=</span><span class="name">np</span><span class="operator">.</span><span class="name">int</span><span class="punctuation">)</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;X = X + 2.0*Y&quot;</span><span class="punctuation">,</span> <span class="name builtin">globals</span><span class="punctuation">())</span>
<span class="generic output">100 loops, best of 3: 3.61 ms per loop
</span><span class="name"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;X = X + 2*Y&quot;</span><span class="punctuation">,</span> <span class="name builtin">globals</span><span class="punctuation">())</span>
<span class="generic output">100 loops, best of 3: 3.47 ms per loop
</span><span class="name"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;X += 2*Y&quot;</span><span class="punctuation">,</span> <span class="name builtin">globals</span><span class="punctuation">())</span>
<span class="generic output">100 loops, best of 3: 2.79 ms per loop
</span><span class="name"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;np.add(X, Y, out=X); np.add(X, Y, out=X)&quot;</span><span class="punctuation">,</span> <span class="name builtin">globals</span><span class="punctuation">())</span>
<span class="generic output">1000 loops, best of 3: 1.57 ms per loop</span>
</pre>
</div>
</div>
<div class="section" id="conclusion">
<h2><a class="toc-backref" href="#id4">Conclusion</a></h2>
<p>As a conclusion, we'll make an exercise. Given two vectors <code>Z1</code> and <code>Z2</code>. We
would like to know if <code>Z2</code> is a view of <code>Z1</code> and if yes, what is this view ?</p>
<pre class="code literal-block">
&gt;&gt;&gt; Z1 = np.arange(10)
&gt;&gt;&gt; Z2 = Z1[1:-1:2]
</pre>
<pre class="code output literal-block">
   ╌╌╌┬───┬───┬───┬───┬───┬───┬───┬───┬───┬───┬╌╌
Z1    │ 0 │ 1 │ 2 │ 3 │ 4 │ 5 │ 6 │ 7 │ 8 │ 9 │
   ╌╌╌┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴╌╌
   ╌╌╌╌╌╌╌┬───┬╌╌╌┬───┬╌╌╌┬───┬╌╌╌┬───┬╌╌╌╌╌╌╌╌╌╌
Z2        │ 1 │   │ 3 │   │ 5 │   │ 7 │
   ╌╌╌╌╌╌╌┴───┴╌╌╌┴───┴╌╌╌┴───┴╌╌╌┴───┴╌╌╌╌╌╌╌╌╌╌
</pre>
<p>First, we need to check if <code>Z1</code> is the base of <code>Z2</code></p>
<pre class="code literal-block">
&gt;&gt;&gt; print(Z2.base is Z1)
True
</pre>
<p>At this point, we know <code>Z2</code> is a view of <code>Z1</code>, meaning <code>Z2</code> can be expressed as
<code>Z1[start:stop:step]</code>. The difficulty is to find <code>start</code>, <code>stop</code> and
<code>step</code>.  For the <code>step</code>, we can use the <code>strides</code> property of any array that
gives the number of bytes to go from one element to the other in each
dimension. In our case, and because both arrays are one-dimensional, we can
directly compare the first stride only:</p>
<pre class="code literal-block">
&gt;&gt;&gt; step = Z2.strides[0] // Z1.strides[0]
&gt;&gt;&gt; print(step)
2
</pre>
<p>Next difficulty is to find the <code>start</code> and the <code>stop</code> indices. To do this, we
can take advantage of the <code>byte_bounds</code> method that returns a pointer to the
end-points of an array.</p>
<pre class="code output literal-block">
  byte_bounds(Z1)[0]                  byte_bounds(Z1)[-1]
          ↓                                   ↓
   ╌╌╌┬───┬───┬───┬───┬───┬───┬───┬───┬───┬───┬╌╌
Z1    │ 0 │ 1 │ 2 │ 3 │ 4 │ 5 │ 6 │ 7 │ 8 │ 9 │
   ╌╌╌┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴╌╌

      byte_bounds(Z2)[0]      byte_bounds(Z2)[-1]
              ↓                       ↓
   ╌╌╌╌╌╌╌┬───┬╌╌╌┬───┬╌╌╌┬───┬╌╌╌┬───┬╌╌╌╌╌╌╌╌╌╌
Z2        │ 1 │   │ 3 │   │ 5 │   │ 7 │
   ╌╌╌╌╌╌╌┴───┴╌╌╌┴───┴╌╌╌┴───┴╌╌╌┴───┴╌╌╌╌╌╌╌╌╌╌
</pre>
<pre class="code literal-block">
&gt;&gt;&gt; offset_start = np.byte_bounds(Z2)[0] - np.byte_bounds(Z1)[0]
&gt;&gt;&gt; print(offset_start) # bytes
8

&gt;&gt;&gt; offset_stop = np.byte_bounds(Z2)[-1] - np.byte_bounds(Z1)[-1]
&gt;&gt;&gt; print(offset_stop) # bytes
-16
</pre>
<p>Converting these offsets into indices is straightforward using the <code>itemsize</code>
and taking into account that the <code>offset_stop</code> is negative (end-bound of <code>Z2</code>
is logically smaller than end-bound of <code>Z1</code> array). We thus need to add the
items size of Z1 to get the right end index.</p>
<pre class="code literal-block">
&gt;&gt;&gt; start = offset_start // Z1.itemsize
&gt;&gt;&gt; stop = Z1.size + offset_stop // Z1.itemsize
&gt;&gt;&gt; print(start, stop, step)
1, 8, 2
</pre>
<p>Last we test our results:</p>
<pre class="code literal-block">
&gt;&gt;&gt; print(np.allclose(Z1[start:stop:step], Z2))
True
</pre>
<p>As an exercise, you can improve this first and very simple implementation by
taking into account:</p>
<ul class="simple">
<li>Negative steps</li>
<li>Multi-dimensional arrays</li>
</ul>
<p><a class="reference external" href="code/find_index.py">Solution</a> to the exercise.</p>
</div>
</div>
<div class="section" id="code-vectorization">
<h1><a class="toc-backref" href="#table-of-contents">Code vectorization</a></h1>
<div class="contents local topic" id="id6">
<p class="topic-title first"><strong>Contents</strong></p>
<ul class="simple">
<li><a class="reference internal" href="#id7" id="id122">Introduction</a></li>
<li><a class="reference internal" href="#uniform-vectorization" id="id123">Uniform vectorization</a><ul>
<li><a class="reference internal" href="#the-game-of-life" id="id124">The Game of Life</a></li>
<li><a class="reference internal" href="#python-implementation" id="id125">Python implementation</a></li>
<li><a class="reference internal" href="#numpy-implementation" id="id126">NumPy implementation</a></li>
<li><a class="reference internal" href="#exercise" id="id127">Exercise</a></li>
<li><a class="reference internal" href="#sources" id="id128">Sources</a></li>
<li><a class="reference internal" href="#references" id="id129">References</a></li>
</ul>
</li>
<li><a class="reference internal" href="#temporal-vectorization" id="id130">Temporal vectorization</a><ul>
<li><a class="reference internal" href="#id8" id="id131">Python implementation</a></li>
<li><a class="reference internal" href="#id9" id="id132">NumPy implementation</a></li>
<li><a class="reference internal" href="#faster-numpy-implementation" id="id133">Faster NumPy implementation</a></li>
<li><a class="reference internal" href="#visualization" id="id134">Visualization</a></li>
<li><a class="reference internal" href="#id10" id="id135">Exercise</a></li>
<li><a class="reference internal" href="#id11" id="id136">Sources</a></li>
<li><a class="reference internal" href="#id12" id="id137">References</a></li>
</ul>
</li>
<li><a class="reference internal" href="#spatial-vectorization" id="id138">Spatial vectorization</a><ul>
<li><a class="reference internal" href="#boids" id="id139">Boids</a></li>
<li><a class="reference internal" href="#id14" id="id140">Python implementation</a></li>
<li><a class="reference internal" href="#id15" id="id141">NumPy implementation</a></li>
<li><a class="reference internal" href="#id16" id="id142">Exercise</a></li>
<li><a class="reference internal" href="#id17" id="id143">Sources</a></li>
<li><a class="reference internal" href="#id18" id="id144">References</a></li>
</ul>
</li>
<li><a class="reference internal" href="#id19" id="id145">Conclusion</a></li>
</ul>
</div>
<div class="section" id="id7">
<h2><a class="toc-backref" href="#id6">Introduction</a></h2>
<p>Code vectorization means that the problem you're trying to solve is inherently
vectorizable and only requires a few NumPy tricks to make it faster. Of course
it does not mean it is easy or straightforward, but at least it does not
necessitate totally rethinking your problem (as it will be the case in the
<a class="reference internal" href="#problem-vectorization">Problem vectorization</a> chapter). Still, it may require some experience to see
where code can be vectorized. Let's illustrate this through a simple example
where we want to sum up two lists of integers. One simple way using pure Python
is:</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function">add_python</span><span class="punctuation">(</span><span class="name">Z1</span><span class="punctuation">,</span><span class="name">Z2</span><span class="punctuation">):</span>
    <span class="keyword">return</span> <span class="punctuation">[</span><span class="name">z1</span><span class="operator">+</span><span class="name">z2</span> <span class="keyword">for</span> <span class="punctuation">(</span><span class="name">z1</span><span class="punctuation">,</span><span class="name">z2</span><span class="punctuation">)</span> <span class="operator word">in</span> <span class="name builtin">zip</span><span class="punctuation">(</span><span class="name">Z1</span><span class="punctuation">,</span><span class="name">Z2</span><span class="punctuation">)]</span>
</pre>
<p>This first naive solution can be vectorized very easily using NumPy:</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function">add_numpy</span><span class="punctuation">(</span><span class="name">Z1</span><span class="punctuation">,</span><span class="name">Z2</span><span class="punctuation">):</span>
    <span class="keyword">return</span> <span class="name">np</span><span class="operator">.</span><span class="name">add</span><span class="punctuation">(</span><span class="name">Z1</span><span class="punctuation">,</span><span class="name">Z2</span><span class="punctuation">)</span>
</pre>
<p>Without any surprise, benchmarking the two approaches shows the second method
is the fastest with one order of magnitude.</p>
<pre class="code python literal-block">
<span class="operator">&gt;&gt;&gt;</span> <span class="name">Z1</span> <span class="operator">=</span> <span class="name">random</span><span class="operator">.</span><span class="name">sample</span><span class="punctuation">(</span><span class="name builtin">range</span><span class="punctuation">(</span><span class="literal number integer">1000</span><span class="punctuation">),</span> <span class="literal number integer">100</span><span class="punctuation">)</span>
<span class="operator">&gt;&gt;&gt;</span> <span class="name">Z2</span> <span class="operator">=</span> <span class="name">random</span><span class="operator">.</span><span class="name">sample</span><span class="punctuation">(</span><span class="name builtin">range</span><span class="punctuation">(</span><span class="literal number integer">1000</span><span class="punctuation">),</span> <span class="literal number integer">100</span><span class="punctuation">)</span>
<span class="operator">&gt;&gt;&gt;</span> <span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;add_python(Z1, Z2)&quot;</span><span class="punctuation">,</span> <span class="name builtin">globals</span><span class="punctuation">())</span>
<span class="literal number integer">1000</span> <span class="name">loops</span><span class="punctuation">,</span> <span class="name">best</span> <span class="name">of</span> <span class="literal number integer">3</span><span class="punctuation">:</span> <span class="literal number integer">68</span> <span class="name">usec</span> <span class="name">per</span> <span class="name">loop</span>
<span class="operator">&gt;&gt;&gt;</span> <span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;add_numpy(Z1, Z2)&quot;</span><span class="punctuation">,</span> <span class="name builtin">globals</span><span class="punctuation">())</span>
<span class="literal number integer">10000</span> <span class="name">loops</span><span class="punctuation">,</span> <span class="name">best</span> <span class="name">of</span> <span class="literal number integer">3</span><span class="punctuation">:</span> <span class="literal number float">1.14</span> <span class="name">usec</span> <span class="name">per</span> <span class="name">loop</span>
</pre>
<p>Not only is the second approach faster, but it also naturally adapts to the
shape of <code>Z1</code> and <code>Z2</code>. This is the reason why we did not write <code>Z1 + Z2</code>
because it would not work if <code>Z1</code> and <code>Z2</code> were both lists. In the first Python
method, the inner <code>+</code> is interpreted differently depending on the nature of the
two objects such that if we consider two nested lists, we get the following
outputs:</p>
<pre class="code pycon literal-block">
<span class="name"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="name">Z1</span> <span class="operator">=</span> <span class="punctuation">[[</span><span class="literal number integer">1</span><span class="punctuation">,</span> <span class="literal number integer">2</span><span class="punctuation">],</span> <span class="punctuation">[</span><span class="literal number integer">3</span><span class="punctuation">,</span> <span class="literal number integer">4</span><span class="punctuation">]]</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="name">Z2</span> <span class="operator">=</span> <span class="punctuation">[[</span><span class="literal number integer">5</span><span class="punctuation">,</span> <span class="literal number integer">6</span><span class="punctuation">],</span> <span class="punctuation">[</span><span class="literal number integer">7</span><span class="punctuation">,</span> <span class="literal number integer">8</span><span class="punctuation">]]</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="name">Z1</span> <span class="operator">+</span> <span class="name">Z2</span>
<span class="generic output">[[1, 2], [3, 4], [5, 6], [7, 8]]
</span><span class="name"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="name">add_python</span><span class="punctuation">(</span><span class="name">Z1</span><span class="punctuation">,</span> <span class="name">Z2</span><span class="punctuation">)</span>
<span class="generic output">[[1, 2, 5, 6], [3, 4, 7, 8]]
</span><span class="name"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="name">add_numpy</span><span class="punctuation">(</span><span class="name">Z1</span><span class="punctuation">,</span> <span class="name">Z2</span><span class="punctuation">)</span>
<span class="generic output">[[ 6  8]
 [10 12]]</span>
</pre>
<p>The first method concatenates the two lists together, the second method
concatenates the internal lists together and the last one computes what is
(numerically) expected. As an exercise, you can rewrite the Python version
such that it accepts nested lists of any depth.</p>
</div>
<div class="section" id="uniform-vectorization">
<h2><a class="toc-backref" href="#id6">Uniform vectorization</a></h2>
<p>Uniform vectorization is the simplest form of vectorization where all the
elements share the same computation at every time step with no specific
processing for any element. One stereotypical case is the Game of Life that has
been invented by John Conway (see below) and is one of the earliest examples of
cellular automata. Those cellular automata can be conveniently regarded as
an array of cells that are connected together with the notion of neighbours and
their vectorization is straightforward. Let me first define the game and we'll
see how to vectorize it.</p>
<div class="admonition legend">
<p class="first admonition-title"><strong>Figure 4.1</strong></p>
<p class="last">Conus textile snail exhibits a cellular automaton pattern on its shell.
Image by <a class="reference external" href="https://commons.wikimedia.org/wiki/File:Textile_cone.JPG">Richard Ling</a>, 2005.</p>
</div>
<img alt="data/Textile-Cone-cropped.jpg" class="bordered" src="data/Textile-Cone-cropped.jpg" style="width: 100%;" />
<div class="section" id="the-game-of-life">
<h3><a class="toc-backref" href="#id6">The Game of Life</a></h3>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">Excerpt from the Wikipedia entry on the
<a class="reference external" href="https://en.wikipedia.org/wiki/Conway%27s_Game_of_Life">Game of Life</a></p>
</div>
<p>The Game of Life is a cellular automaton devised by the British mathematician
John Horton Conway in 1970. It is the best-known example of a cellular
automaton. The &quot;game&quot; is actually a zero-player game, meaning that its
evolution is determined by its initial state, needing no input from human
players. One interacts with the Game of Life by creating an initial
configuration and observing how it evolves.</p>
<p>The universe of the Game of Life is an infinite two-dimensional orthogonal grid
of square cells, each of which is in one of two possible states, live or
dead. Every cell interacts with its eight neighbours, which are the cells that
are directly horizontally, vertically, or diagonally adjacent. At each step in
time, the following transitions occur:</p>
<ol class="arabic simple">
<li>Any live cell with fewer than two live neighbours dies, as if by needs
caused by underpopulation.</li>
<li>Any live cell with more than three live neighbours dies, as if by
overcrowding.</li>
<li>Any live cell with two or three live neighbours lives, unchanged, to the
next generation.</li>
<li>Any dead cell with exactly three live neighbours becomes a live cell.</li>
</ol>
<p>The initial pattern constitutes the 'seed' of the system. The first generation
is created by applying the above rules simultaneously to every cell in the seed
– births and deaths happen simultaneously, and the discrete moment at which
this happens is sometimes called a tick. (In other words, each generation is a
pure function of the one before.) The rules continue to be applied repeatedly
to create further generations.</p>
</div>
<div class="section" id="python-implementation">
<h3><a class="toc-backref" href="#id6">Python implementation</a></h3>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">We could have used the more efficient python <a class="reference external" href="http://docs.python.org/3/library/array.html">array interface</a> but it is more convenient to
use the familiar list object.</p>
</div>
<p>In pure Python, we can code the Game of Life using a list of lists representing
the board where cells are supposed to evolve. Such a board will be equipped with
border of 0 that allows to accelerate things a bit by avoiding having specific
tests for borders when counting the number of neighbours.</p>
<pre class="code python literal-block">
<span class="name">Z</span> <span class="operator">=</span> <span class="punctuation">[[</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">],</span>
     <span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">],</span>
     <span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">],</span>
     <span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">],</span>
     <span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">],</span>
     <span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">]]</span>
</pre>
<p>Taking the border into account, counting neighbours then is straightforward:</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function">compute_neighbours</span><span class="punctuation">(</span><span class="name">Z</span><span class="punctuation">):</span>
    <span class="name">shape</span> <span class="operator">=</span> <span class="name builtin">len</span><span class="punctuation">(</span><span class="name">Z</span><span class="punctuation">),</span> <span class="name builtin">len</span><span class="punctuation">(</span><span class="name">Z</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">])</span>
    <span class="name">N</span>  <span class="operator">=</span> <span class="punctuation">[[</span><span class="literal number integer">0</span><span class="punctuation">,]</span><span class="operator">*</span><span class="punctuation">(</span><span class="name">shape</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">])</span> <span class="keyword">for</span> <span class="name">i</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="name">shape</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">])]</span>
    <span class="keyword">for</span> <span class="name">x</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="name">shape</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">]</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">):</span>
        <span class="keyword">for</span> <span class="name">y</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="name">shape</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">]</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">):</span>
            <span class="name">N</span><span class="punctuation">[</span><span class="name">x</span><span class="punctuation">][</span><span class="name">y</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="name">Z</span><span class="punctuation">[</span><span class="name">x</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">][</span><span class="name">y</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">]</span><span class="operator">+</span><span class="name">Z</span><span class="punctuation">[</span><span class="name">x</span><span class="punctuation">][</span><span class="name">y</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">]</span><span class="operator">+</span><span class="name">Z</span><span class="punctuation">[</span><span class="name">x</span><span class="operator">+</span><span class="literal number integer">1</span><span class="punctuation">][</span><span class="name">y</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">]</span> \
                    <span class="operator">+</span> <span class="name">Z</span><span class="punctuation">[</span><span class="name">x</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">][</span><span class="name">y</span><span class="punctuation">]</span>            <span class="operator">+</span><span class="name">Z</span><span class="punctuation">[</span><span class="name">x</span><span class="operator">+</span><span class="literal number integer">1</span><span class="punctuation">][</span><span class="name">y</span><span class="punctuation">]</span>   \
                    <span class="operator">+</span> <span class="name">Z</span><span class="punctuation">[</span><span class="name">x</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">][</span><span class="name">y</span><span class="operator">+</span><span class="literal number integer">1</span><span class="punctuation">]</span><span class="operator">+</span><span class="name">Z</span><span class="punctuation">[</span><span class="name">x</span><span class="punctuation">][</span><span class="name">y</span><span class="operator">+</span><span class="literal number integer">1</span><span class="punctuation">]</span><span class="operator">+</span><span class="name">Z</span><span class="punctuation">[</span><span class="name">x</span><span class="operator">+</span><span class="literal number integer">1</span><span class="punctuation">][</span><span class="name">y</span><span class="operator">+</span><span class="literal number integer">1</span><span class="punctuation">]</span>
    <span class="keyword">return</span> <span class="name">N</span>
</pre>
<p>To iterate one step in time, we then simply count the number of neighbours for
each internal cell and we update the whole board according to the four
aforementioned rules:</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function">iterate</span><span class="punctuation">(</span><span class="name">Z</span><span class="punctuation">):</span>
    <span class="name">N</span> <span class="operator">=</span> <span class="name">compute_neighbours</span><span class="punctuation">(</span><span class="name">Z</span><span class="punctuation">)</span>
    <span class="keyword">for</span> <span class="name">x</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="name">shape</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">]</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">):</span>
        <span class="keyword">for</span> <span class="name">y</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="name">shape</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">]</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">):</span>
             <span class="keyword">if</span> <span class="name">Z</span><span class="punctuation">[</span><span class="name">x</span><span class="punctuation">][</span><span class="name">y</span><span class="punctuation">]</span> <span class="operator">==</span> <span class="literal number integer">1</span> <span class="operator word">and</span> <span class="punctuation">(</span><span class="name">N</span><span class="punctuation">[</span><span class="name">x</span><span class="punctuation">][</span><span class="name">y</span><span class="punctuation">]</span> <span class="operator">&lt;</span> <span class="literal number integer">2</span> <span class="operator word">or</span> <span class="name">N</span><span class="punctuation">[</span><span class="name">x</span><span class="punctuation">][</span><span class="name">y</span><span class="punctuation">]</span> <span class="operator">&gt;</span> <span class="literal number integer">3</span><span class="punctuation">):</span>
                 <span class="name">Z</span><span class="punctuation">[</span><span class="name">x</span><span class="punctuation">][</span><span class="name">y</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="literal number integer">0</span>
             <span class="keyword">elif</span> <span class="name">Z</span><span class="punctuation">[</span><span class="name">x</span><span class="punctuation">][</span><span class="name">y</span><span class="punctuation">]</span> <span class="operator">==</span> <span class="literal number integer">0</span> <span class="operator word">and</span> <span class="name">N</span><span class="punctuation">[</span><span class="name">x</span><span class="punctuation">][</span><span class="name">y</span><span class="punctuation">]</span> <span class="operator">==</span> <span class="literal number integer">3</span><span class="punctuation">:</span>
                 <span class="name">Z</span><span class="punctuation">[</span><span class="name">x</span><span class="punctuation">][</span><span class="name">y</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="literal number integer">1</span>
    <span class="keyword">return</span> <span class="name">Z</span>
</pre>
<p>The figure below shows four iterations on a 4x4 area where the initial state is a
<a class="reference external" href="https://en.wikipedia.org/wiki/Glider_(Conway%27s_Life)">glider</a>, a structure
discovered by Richard K. Guy in 1970.</p>
<div class="admonition legend">
<p class="first admonition-title"><strong>Figure 4.2</strong></p>
<p class="last">The glider pattern is known to replicate itself one step diagonally in 4
iterations.</p>
</div>
<img alt="data/glider.png" src="data/glider.png" style="width: 100%;" />
</div>
<div class="section" id="numpy-implementation">
<h3><a class="toc-backref" href="#id6">NumPy implementation</a></h3>
<p>Starting from the Python version, the vectorization of the Game of Life
requires two parts, one responsible for counting the neighbours and one
responsible for enforcing the rules. Neighbour-counting is relatively easy if
we remember we took care of adding a null border around the arena. By
considering partial views of the arena we can actually access neighbours quite
intuitively as illustrated below for the one-dimensional case:</p>
<pre class="code output literal-block">
               ┏━━━┳━━━┳━━━┓───┬───┐
        Z[:-2] ┃ 0 ┃ 1 ┃ 1 ┃ 1 │ 0 │ (left neighbours)
               ┗━━━┻━━━┻━━━┛───┴───┘
                     ↓︎
           ┌───┏━━━┳━━━┳━━━┓───┐
   Z[1:-1] │ 0 ┃ 1 ┃ 1 ┃ 1 ┃ 0 │ (actual cells)
           └───┗━━━┻━━━┻━━━┛───┘
                     ↑
       ┌───┬───┏━━━┳━━━┳━━━┓
Z[+2:] │ 0 │ 1 ┃ 1 ┃ 1 ┃ 0 ┃ (right neighbours)
       └───┴───┗━━━┻━━━┻━━━┛
</pre>
<p>Going to the two dimensional case requires just a bit of arithmetic to make
sure to consider all the eight neighbours.</p>
<pre class="code python literal-block">
<span class="name">N</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">zeros</span><span class="punctuation">(</span><span class="name">Z</span><span class="operator">.</span><span class="name">shape</span><span class="punctuation">,</span> <span class="name">dtype</span><span class="operator">=</span><span class="name builtin">int</span><span class="punctuation">)</span>
<span class="name">N</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">]</span> <span class="operator">+=</span> <span class="punctuation">(</span><span class="name">Z</span><span class="punctuation">[</span> <span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">2</span><span class="punctuation">,</span> <span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">2</span><span class="punctuation">]</span> <span class="operator">+</span> <span class="name">Z</span><span class="punctuation">[</span> <span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">2</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">]</span> <span class="operator">+</span> <span class="name">Z</span><span class="punctuation">[</span> <span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">2</span><span class="punctuation">,</span><span class="literal number integer">2</span><span class="punctuation">:]</span> <span class="operator">+</span>
                 <span class="name">Z</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">,</span> <span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">2</span><span class="punctuation">]</span>                <span class="operator">+</span> <span class="name">Z</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">2</span><span class="punctuation">:]</span> <span class="operator">+</span>
                 <span class="name">Z</span><span class="punctuation">[</span><span class="literal number integer">2</span><span class="punctuation">:</span>  <span class="punctuation">,</span> <span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">2</span><span class="punctuation">]</span> <span class="operator">+</span> <span class="name">Z</span><span class="punctuation">[</span><span class="literal number integer">2</span><span class="punctuation">:</span>  <span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">]</span> <span class="operator">+</span> <span class="name">Z</span><span class="punctuation">[</span><span class="literal number integer">2</span><span class="punctuation">:</span>  <span class="punctuation">,</span><span class="literal number integer">2</span><span class="punctuation">:])</span>
</pre>
<p>For the rule enforcement, we can write a first version using NumPy's
<a class="reference external" href="http://docs.scipy.org/doc/numpy/reference/generated/numpy.argwhere.html">argwhere</a>
method that will give us the indices where a given condition is True.</p>
<pre class="code python literal-block">
<span class="comment single"># Flatten arrays</span>
<span class="name">N_</span> <span class="operator">=</span> <span class="name">N</span><span class="operator">.</span><span class="name">ravel</span><span class="punctuation">()</span>
<span class="name">Z_</span> <span class="operator">=</span> <span class="name">Z</span><span class="operator">.</span><span class="name">ravel</span><span class="punctuation">()</span>

<span class="comment single"># Apply rules</span>
<span class="name">R1</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">argwhere</span><span class="punctuation">(</span> <span class="punctuation">(</span><span class="name">Z_</span><span class="operator">==</span><span class="literal number integer">1</span><span class="punctuation">)</span> <span class="operator">&amp;</span> <span class="punctuation">(</span><span class="name">N_</span> <span class="operator">&lt;</span> <span class="literal number integer">2</span><span class="punctuation">)</span> <span class="punctuation">)</span>
<span class="name">R2</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">argwhere</span><span class="punctuation">(</span> <span class="punctuation">(</span><span class="name">Z_</span><span class="operator">==</span><span class="literal number integer">1</span><span class="punctuation">)</span> <span class="operator">&amp;</span> <span class="punctuation">(</span><span class="name">N_</span> <span class="operator">&gt;</span> <span class="literal number integer">3</span><span class="punctuation">)</span> <span class="punctuation">)</span>
<span class="name">R3</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">argwhere</span><span class="punctuation">(</span> <span class="punctuation">(</span><span class="name">Z_</span><span class="operator">==</span><span class="literal number integer">1</span><span class="punctuation">)</span> <span class="operator">&amp;</span> <span class="punctuation">((</span><span class="name">N_</span><span class="operator">==</span><span class="literal number integer">2</span><span class="punctuation">)</span> <span class="operator">|</span> <span class="punctuation">(</span><span class="name">N_</span><span class="operator">==</span><span class="literal number integer">3</span><span class="punctuation">))</span> <span class="punctuation">)</span>
<span class="name">R4</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">argwhere</span><span class="punctuation">(</span> <span class="punctuation">(</span><span class="name">Z_</span><span class="operator">==</span><span class="literal number integer">0</span><span class="punctuation">)</span> <span class="operator">&amp;</span> <span class="punctuation">(</span><span class="name">N_</span><span class="operator">==</span><span class="literal number integer">3</span><span class="punctuation">)</span> <span class="punctuation">)</span>

<span class="comment single"># Set new values</span>
<span class="name">Z_</span><span class="punctuation">[</span><span class="name">R1</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="literal number integer">0</span>
<span class="name">Z_</span><span class="punctuation">[</span><span class="name">R2</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="literal number integer">0</span>
<span class="name">Z_</span><span class="punctuation">[</span><span class="name">R3</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="name">Z_</span><span class="punctuation">[</span><span class="name">R3</span><span class="punctuation">]</span>
<span class="name">Z_</span><span class="punctuation">[</span><span class="name">R4</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="literal number integer">1</span>

<span class="comment single"># Make sure borders stay null</span>
<span class="name">Z</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">,:]</span> <span class="operator">=</span> <span class="name">Z</span><span class="punctuation">[</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">,:]</span> <span class="operator">=</span> <span class="name">Z</span><span class="punctuation">[:,</span><span class="literal number integer">0</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="name">Z</span><span class="punctuation">[:,</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="literal number integer">0</span>
</pre>
<p>Even if this first version does not use nested loops, it is far from optimal
because of the use of the four <code>argwhere</code> calls that may be quite slow. We can
instead factorize the rules into cells that will survive (stay at 1) and cells
that will give birth. For doing this, we can take advantage of NumPy boolean
capability and write quite naturally:</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">We did not write <code>Z = 0</code> as this would simply assign the value 0 to <code>Z</code> that
would then become a simple scalar.</p>
</div>
<pre class="code python literal-block">
<span class="name">birth</span> <span class="operator">=</span> <span class="punctuation">(</span><span class="name">N</span><span class="operator">==</span><span class="literal number integer">3</span><span class="punctuation">)[</span><span class="literal number integer">1</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">]</span> <span class="operator">&amp;</span> <span class="punctuation">(</span><span class="name">Z</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">]</span><span class="operator">==</span><span class="literal number integer">0</span><span class="punctuation">)</span>
<span class="name">survive</span> <span class="operator">=</span> <span class="punctuation">((</span><span class="name">N</span><span class="operator">==</span><span class="literal number integer">2</span><span class="punctuation">)</span> <span class="operator">|</span> <span class="punctuation">(</span><span class="name">N</span><span class="operator">==</span><span class="literal number integer">3</span><span class="punctuation">))[</span><span class="literal number integer">1</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">]</span> <span class="operator">&amp;</span> <span class="punctuation">(</span><span class="name">Z</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">]</span><span class="operator">==</span><span class="literal number integer">1</span><span class="punctuation">)</span>
<span class="name">Z</span><span class="punctuation">[</span><span class="operator">...</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="literal number integer">0</span>
<span class="name">Z</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">][</span><span class="name">birth</span> <span class="operator">|</span> <span class="name">survive</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="literal number integer">1</span>
</pre>
<p>If you look at the <code>birth</code> and <code>survive</code> lines, you'll see that these two
variables are arrays that can be used to set <code>Z</code> values to 1 after having
cleared it.</p>
<div class="admonition legend">
<p class="first admonition-title"><strong>Figure 4.3</strong></p>
<p class="last">The Game of Life. Gray levels indicate how much a cell has been active in
the past.</p>
</div>
<video width="100%" class="bordered" controls>
<source src="data/game-of-life.mp4" type="video/mp4">
Your browser does not support the video tag. </video></div>
<div class="section" id="exercise">
<h3><a class="toc-backref" href="#id6">Exercise</a></h3>
<p>Reaction and diffusion of chemical species can produce a variety of
patterns, reminiscent of those often seen in nature. The Gray-Scott
equations model such a reaction. For more information on this chemical
system see the article <em>Complex Patterns in a Simple System</em>
(John E. Pearson, Science, Volume 261, 1993). Let's consider two
chemical species <span class="formula"><i>U</i></span> and <span class="formula"><i>V</i></span> with respective
concentrations <span class="formula"><i>u</i></span> and <span class="formula"><i>v</i></span> and diffusion rates <span class="formula"><i>Du</i></span>
and <span class="formula"><i>Dv</i></span>. <span class="formula"><i>V</i></span> is converted into <span class="formula"><i>P</i></span> with a rate of
conversion <span class="formula"><i>k</i></span>. <span class="formula"><i>f</i></span> represents the rate of the process
that feeds <span class="formula"><i>U</i></span> and drains <span class="formula"><i>U</i></span>, <span class="formula"><i>V</i></span> and
<span class="formula"><i>P</i></span>. This can be written as:</p>
<table border="1" class="colwidths-given docutils">
<colgroup>
<col width="50%" />
<col width="50%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">Chemical reaction</th>
<th class="head">Equations</th>
</tr>
</thead>
<tbody valign="top">
<tr><td><span class="formula"><i>U</i> + 2<i>V</i> → 3<i>V</i></span></td>
<td><span class="formula"><i>u̇</i> = <i>Du</i>∇<sup>2</sup><i>u</i> − <i>uv</i><sup>2</sup> + <i>f</i>(1 − <i>u</i>)</span></td>
</tr>
<tr><td><span class="formula"><i>V</i> → <i>P</i></span></td>
<td><span class="formula"><i>v̇</i> = <i>Dv</i>∇<sup>2</sup><i>v</i> + <i>uv</i><sup>2</sup> − (<i>f</i> + <i>k</i>)<i>v</i></span></td>
</tr>
</tbody>
</table>
<p>Based on the Game of Life example, try to implement such reaction-diffusion system.
Here is a set of interesting parameters to test:</p>
<table border="1" class="docutils">
<colgroup>
<col width="39%" />
<col width="15%" />
<col width="15%" />
<col width="15%" />
<col width="15%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">Name</th>
<th class="head">Du</th>
<th class="head">Dv</th>
<th class="head">f</th>
<th class="head">k</th>
</tr>
</thead>
<tbody valign="top">
<tr><td>Bacteria 1</td>
<td>0.16</td>
<td>0.08</td>
<td>0.035</td>
<td>0.065</td>
</tr>
<tr><td>Bacteria 2</td>
<td>0.14</td>
<td>0.06</td>
<td>0.035</td>
<td>0.065</td>
</tr>
<tr><td>Coral</td>
<td>0.16</td>
<td>0.08</td>
<td>0.060</td>
<td>0.062</td>
</tr>
<tr><td>Fingerprint</td>
<td>0.19</td>
<td>0.05</td>
<td>0.060</td>
<td>0.062</td>
</tr>
<tr><td>Spirals</td>
<td>0.10</td>
<td>0.10</td>
<td>0.018</td>
<td>0.050</td>
</tr>
<tr><td>Spirals Dense</td>
<td>0.12</td>
<td>0.08</td>
<td>0.020</td>
<td>0.050</td>
</tr>
<tr><td>Spirals Fast</td>
<td>0.10</td>
<td>0.16</td>
<td>0.020</td>
<td>0.050</td>
</tr>
<tr><td>Unstable</td>
<td>0.16</td>
<td>0.08</td>
<td>0.020</td>
<td>0.055</td>
</tr>
<tr><td>Worms 1</td>
<td>0.16</td>
<td>0.08</td>
<td>0.050</td>
<td>0.065</td>
</tr>
<tr><td>Worms 2</td>
<td>0.16</td>
<td>0.08</td>
<td>0.054</td>
<td>0.063</td>
</tr>
<tr><td>Zebrafish</td>
<td>0.16</td>
<td>0.08</td>
<td>0.035</td>
<td>0.060</td>
</tr>
</tbody>
</table>
<p>The figure below shows some animations of the model for a specific set of parameters.</p>
<div class="admonition legend">
<p class="first admonition-title"><strong>Figure 4.4</strong></p>
<p class="last">Reaction-diffusion Gray-Scott model. From left to right, <em>Bacteria 1</em>, <em>Coral</em> and
<em>Spiral Dense</em>.</p>
</div>
<video width="33%" controls>
<source src="data/gray-scott-1.mp4" type="video/mp4">
Your browser does not support the video tag. </video>

<video width="33%" controls>
<source src="data/gray-scott-2.mp4" type="video/mp4">
Your browser does not support the video tag. </video>

<video width="33%" controls>
<source src="data/gray-scott-3.mp4" type="video/mp4">
Your browser does not support the video tag. </video></div>
<div class="section" id="sources">
<h3><a class="toc-backref" href="#id6">Sources</a></h3>
<ul class="simple">
<li><a class="reference external" href="code/game_of_life_python.py">game_of_life_python.py</a></li>
<li><a class="reference external" href="code/game_of_life_numpy.py">game_of_life_numpy.py</a></li>
<li><a class="reference external" href="code/gray_scott.py">gray_scott.py</a> (solution to the exercise)</li>
</ul>
</div>
<div class="section" id="references">
<h3><a class="toc-backref" href="#id6">References</a></h3>
<ul class="simple">
<li><a class="reference external" href="https://web.archive.org/web/20090603015231/http://ddi.cs.uni-potsdam.de/HyFISCH/Produzieren/lis_projekt/proj_gamelife/ConwayScientificAmerican.htm">John Conway new solitaire game &quot;life&quot;</a>, Martin Gardner, Scientific American 223, 1970.</li>
<li><a class="reference external" href="http://groups.csail.mit.edu/mac/projects/amorphous/GrayScott/">Gray Scott Model of Reaction Diffusion</a>, Abelson, Adams, Coore, Hanson, Nagpal, Sussman, 1997.</li>
<li><a class="reference external" href="http://mrob.com/pub/comp/xmorphia/">Reaction-Diffusion by the Gray-Scott Model</a>,
Robert P. Munafo, 1996.</li>
</ul>
</div>
</div>
<div class="section" id="temporal-vectorization">
<h2><a class="toc-backref" href="#id6">Temporal vectorization</a></h2>
<p>The Mandelbrot set is the set of complex numbers <span class="formula"><i>c</i></span> for which
the function <span class="formula"><i>f</i><sub><i>c</i></sub>(<i>z</i>) = <i>z</i><sup>2</sup> + <i>c</i></span> does not diverge when iterated
from <span class="formula"><i>z</i> = 0</span>, i.e., for which the sequence <span class="formula"><i>f</i><sub><i>c</i></sub>(0), <i>f</i><sub><i>c</i></sub>(<i>f</i><sub><i>c</i></sub>(0))</span>, etc., remains bounded in absolute value. It is very easy
to compute, but it can take a very long time because you need to
ensure a given number does not diverge. This is generally done by
iterating the computation up to a maximum number of iterations, after
which, if the number is still within some bounds, it is considered
non-divergent. Of course, the more iterations you do, the more
precision you get.</p>
<div class="admonition legend">
<p class="first admonition-title"><strong>Figure 4.5</strong></p>
<p class="last">Romanesco broccoli, showing self-similar form approximating a natural fractal.
Image by <a class="reference external" href="https://commons.wikimedia.org/wiki/File:Fractal_Broccoli.jpg">Jon Sullivan</a>, 2004.</p>
</div>
<img alt="data/Fractal-Broccoli-cropped.jpg" class="bordered" src="data/Fractal-Broccoli-cropped.jpg" style="width: 100%;" />
<div class="section" id="id8">
<h3><a class="toc-backref" href="#id6">Python implementation</a></h3>
<p>A pure python implementation is written as:</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function">mandelbrot_python</span><span class="punctuation">(</span><span class="name">xmin</span><span class="punctuation">,</span> <span class="name">xmax</span><span class="punctuation">,</span> <span class="name">ymin</span><span class="punctuation">,</span> <span class="name">ymax</span><span class="punctuation">,</span> <span class="name">xn</span><span class="punctuation">,</span> <span class="name">yn</span><span class="punctuation">,</span> <span class="name">maxiter</span><span class="punctuation">,</span> <span class="name">horizon</span><span class="operator">=</span><span class="literal number float">2.0</span><span class="punctuation">):</span>
    <span class="keyword">def</span> <span class="name function">mandelbrot</span><span class="punctuation">(</span><span class="name">z</span><span class="punctuation">,</span> <span class="name">maxiter</span><span class="punctuation">):</span>
        <span class="name">c</span> <span class="operator">=</span> <span class="name">z</span>
        <span class="keyword">for</span> <span class="name">n</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="name">maxiter</span><span class="punctuation">):</span>
            <span class="keyword">if</span> <span class="name builtin">abs</span><span class="punctuation">(</span><span class="name">z</span><span class="punctuation">)</span> <span class="operator">&gt;</span> <span class="name">horizon</span><span class="punctuation">:</span>
                <span class="keyword">return</span> <span class="name">n</span>
            <span class="name">z</span> <span class="operator">=</span> <span class="name">z</span><span class="operator">*</span><span class="name">z</span> <span class="operator">+</span> <span class="name">c</span>
        <span class="keyword">return</span> <span class="name">maxiter</span>
    <span class="name">r1</span> <span class="operator">=</span> <span class="punctuation">[</span><span class="name">xmin</span><span class="operator">+</span><span class="name">i</span><span class="operator">*</span><span class="punctuation">(</span><span class="name">xmax</span><span class="operator">-</span><span class="name">xmin</span><span class="punctuation">)</span><span class="operator">/</span><span class="name">xn</span> <span class="keyword">for</span> <span class="name">i</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="name">xn</span><span class="punctuation">)]</span>
    <span class="name">r2</span> <span class="operator">=</span> <span class="punctuation">[</span><span class="name">ymin</span><span class="operator">+</span><span class="name">i</span><span class="operator">*</span><span class="punctuation">(</span><span class="name">ymax</span><span class="operator">-</span><span class="name">ymin</span><span class="punctuation">)</span><span class="operator">/</span><span class="name">yn</span> <span class="keyword">for</span> <span class="name">i</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="name">yn</span><span class="punctuation">)]</span>
    <span class="keyword">return</span> <span class="punctuation">[</span><span class="name">mandelbrot</span><span class="punctuation">(</span><span class="name builtin">complex</span><span class="punctuation">(</span><span class="name">r</span><span class="punctuation">,</span> <span class="name">i</span><span class="punctuation">),</span><span class="name">maxiter</span><span class="punctuation">)</span> <span class="keyword">for</span> <span class="name">r</span> <span class="operator word">in</span> <span class="name">r1</span> <span class="keyword">for</span> <span class="name">i</span> <span class="operator word">in</span> <span class="name">r2</span><span class="punctuation">]</span>
</pre>
<p>The interesting (and slow) part of this code is the <code>mandelbrot</code> function that
actually computes the sequence <span class="formula"><i>f</i><sub><i>c</i></sub>(<i>f</i><sub><i>c</i></sub>(<i>f</i><sub><i>c</i></sub>...)))</span>. The vectorization of
such code is not totally straightforward because the internal <code>return</code> implies a
differential processing of the element. Once it has diverged, we don't need to
iterate any more and we can safely return the iteration count at
divergence. The problem is to then do the same in NumPy. But how?</p>
</div>
<div class="section" id="id9">
<h3><a class="toc-backref" href="#id6">NumPy implementation</a></h3>
<p>The trick is to search at each iteration values that have not yet
diverged and update relevant information for these values and only
these values. Because we start from <span class="formula"><i>Z</i> = 0</span>, we know that each
value will be updated at least once (when they're equal to <span class="formula">0</span>,
they have not yet diverged) and will stop being updated as soon as
they've diverged. To do that, we'll use NumPy fancy indexing with the
<code>less(x1,x2)</code> function that return the truth value of <code>(x1 &lt; x2)</code>
element-wise.</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function">mandelbrot_numpy</span><span class="punctuation">(</span><span class="name">xmin</span><span class="punctuation">,</span> <span class="name">xmax</span><span class="punctuation">,</span> <span class="name">ymin</span><span class="punctuation">,</span> <span class="name">ymax</span><span class="punctuation">,</span> <span class="name">xn</span><span class="punctuation">,</span> <span class="name">yn</span><span class="punctuation">,</span> <span class="name">maxiter</span><span class="punctuation">,</span> <span class="name">horizon</span><span class="operator">=</span><span class="literal number float">2.0</span><span class="punctuation">):</span>
    <span class="name">X</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">linspace</span><span class="punctuation">(</span><span class="name">xmin</span><span class="punctuation">,</span> <span class="name">xmax</span><span class="punctuation">,</span> <span class="name">xn</span><span class="punctuation">,</span> <span class="name">dtype</span><span class="operator">=</span><span class="name">np</span><span class="operator">.</span><span class="name">float32</span><span class="punctuation">)</span>
    <span class="name">Y</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">linspace</span><span class="punctuation">(</span><span class="name">ymin</span><span class="punctuation">,</span> <span class="name">ymax</span><span class="punctuation">,</span> <span class="name">yn</span><span class="punctuation">,</span> <span class="name">dtype</span><span class="operator">=</span><span class="name">np</span><span class="operator">.</span><span class="name">float32</span><span class="punctuation">)</span>
    <span class="name">C</span> <span class="operator">=</span> <span class="name">X</span> <span class="operator">+</span> <span class="name">Y</span><span class="punctuation">[:,</span><span class="name builtin pseudo">None</span><span class="punctuation">]</span><span class="operator">*</span><span class="literal number integer">1j</span>
    <span class="name">N</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">zeros</span><span class="punctuation">(</span><span class="name">C</span><span class="operator">.</span><span class="name">shape</span><span class="punctuation">,</span> <span class="name">dtype</span><span class="operator">=</span><span class="name builtin">int</span><span class="punctuation">)</span>
    <span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">zeros</span><span class="punctuation">(</span><span class="name">C</span><span class="operator">.</span><span class="name">shape</span><span class="punctuation">,</span> <span class="name">np</span><span class="operator">.</span><span class="name">complex64</span><span class="punctuation">)</span>
    <span class="keyword">for</span> <span class="name">n</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="name">maxiter</span><span class="punctuation">):</span>
        <span class="name">I</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">less</span><span class="punctuation">(</span><span class="name builtin">abs</span><span class="punctuation">(</span><span class="name">Z</span><span class="punctuation">),</span> <span class="name">horizon</span><span class="punctuation">)</span>
        <span class="name">N</span><span class="punctuation">[</span><span class="name">I</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="name">n</span>
        <span class="name">Z</span><span class="punctuation">[</span><span class="name">I</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="name">Z</span><span class="punctuation">[</span><span class="name">I</span><span class="punctuation">]</span><span class="operator">**</span><span class="literal number integer">2</span> <span class="operator">+</span> <span class="name">C</span><span class="punctuation">[</span><span class="name">I</span><span class="punctuation">]</span>
    <span class="name">N</span><span class="punctuation">[</span><span class="name">N</span> <span class="operator">==</span> <span class="name">maxiter</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="literal number integer">0</span>
    <span class="keyword">return</span> <span class="name">Z</span><span class="punctuation">,</span> <span class="name">N</span>
</pre>
<p>Here is the benchmark:</p>
<pre class="code pycon literal-block">
<span class="name"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="name">xmin</span><span class="punctuation">,</span> <span class="name">xmax</span><span class="punctuation">,</span> <span class="name">xn</span> <span class="operator">=</span> <span class="operator">-</span><span class="literal number float">2.25</span><span class="punctuation">,</span> <span class="operator">+</span><span class="literal number float">0.75</span><span class="punctuation">,</span> <span class="name builtin">int</span><span class="punctuation">(</span><span class="literal number integer">3000</span><span class="operator">/</span><span class="literal number integer">3</span><span class="punctuation">)</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="name">ymin</span><span class="punctuation">,</span> <span class="name">ymax</span><span class="punctuation">,</span> <span class="name">yn</span> <span class="operator">=</span> <span class="operator">-</span><span class="literal number float">1.25</span><span class="punctuation">,</span> <span class="operator">+</span><span class="literal number float">1.25</span><span class="punctuation">,</span> <span class="name builtin">int</span><span class="punctuation">(</span><span class="literal number integer">2500</span><span class="operator">/</span><span class="literal number integer">3</span><span class="punctuation">)</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="name">maxiter</span> <span class="operator">=</span> <span class="literal number integer">200</span>
<span class="generic prompt">&gt;&gt;&gt; </span><span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;mandelbrot_python(xmin, xmax, ymin, ymax, xn, yn, maxiter)&quot;</span><span class="punctuation">,</span> <span class="name builtin">globals</span><span class="punctuation">())</span>
<span class="generic output">1 loops, best of 3: 6.1 sec per loop
</span><span class="name"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;mandelbrot_numpy(xmin, xmax, ymin, ymax, xn, yn, maxiter)&quot;</span><span class="punctuation">,</span> <span class="name builtin">globals</span><span class="punctuation">())</span>
<span class="generic output">1 loops, best of 3: 1.15 sec per loop</span>
</pre>
</div>
<div class="section" id="faster-numpy-implementation">
<h3><a class="toc-backref" href="#id6">Faster NumPy implementation</a></h3>
<p>The gain is roughly a 5x factor, not as much as we could have
expected. Part of the problem is that the <code>np.less</code> function implies
<span class="formula"><i>xn</i> × <i>yn</i></span> tests at every iteration while we know that some
values have already diverged. Even if these tests are performed at the
C level (through NumPy), the cost is nonetheless
significant. Another approach proposed by <a class="reference external" href="https://thesamovar.wordpress.com/">Dan Goodman</a> is to work on a dynamic array at
each iteration that stores only the points which have not yet
diverged. It requires more lines but the result is faster and leads to
a 10x factor speed improvement compared to the Python version.</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function">mandelbrot_numpy_2</span><span class="punctuation">(</span><span class="name">xmin</span><span class="punctuation">,</span> <span class="name">xmax</span><span class="punctuation">,</span> <span class="name">ymin</span><span class="punctuation">,</span> <span class="name">ymax</span><span class="punctuation">,</span> <span class="name">xn</span><span class="punctuation">,</span> <span class="name">yn</span><span class="punctuation">,</span> <span class="name">itermax</span><span class="punctuation">,</span> <span class="name">horizon</span><span class="operator">=</span><span class="literal number float">2.0</span><span class="punctuation">):</span>
    <span class="name">Xi</span><span class="punctuation">,</span> <span class="name">Yi</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">mgrid</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">:</span><span class="name">xn</span><span class="punctuation">,</span> <span class="literal number integer">0</span><span class="punctuation">:</span><span class="name">yn</span><span class="punctuation">]</span>
    <span class="name">Xi</span><span class="punctuation">,</span> <span class="name">Yi</span> <span class="operator">=</span> <span class="name">Xi</span><span class="operator">.</span><span class="name">astype</span><span class="punctuation">(</span><span class="name">np</span><span class="operator">.</span><span class="name">uint32</span><span class="punctuation">),</span> <span class="name">Yi</span><span class="operator">.</span><span class="name">astype</span><span class="punctuation">(</span><span class="name">np</span><span class="operator">.</span><span class="name">uint32</span><span class="punctuation">)</span>
    <span class="name">X</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">linspace</span><span class="punctuation">(</span><span class="name">xmin</span><span class="punctuation">,</span> <span class="name">xmax</span><span class="punctuation">,</span> <span class="name">xn</span><span class="punctuation">,</span> <span class="name">dtype</span><span class="operator">=</span><span class="name">np</span><span class="operator">.</span><span class="name">float32</span><span class="punctuation">)[</span><span class="name">Xi</span><span class="punctuation">]</span>
    <span class="name">Y</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">linspace</span><span class="punctuation">(</span><span class="name">ymin</span><span class="punctuation">,</span> <span class="name">ymax</span><span class="punctuation">,</span> <span class="name">yn</span><span class="punctuation">,</span> <span class="name">dtype</span><span class="operator">=</span><span class="name">np</span><span class="operator">.</span><span class="name">float32</span><span class="punctuation">)[</span><span class="name">Yi</span><span class="punctuation">]</span>
    <span class="name">C</span> <span class="operator">=</span> <span class="name">X</span> <span class="operator">+</span> <span class="name">Y</span><span class="operator">*</span><span class="literal number integer">1j</span>
    <span class="name">N_</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">zeros</span><span class="punctuation">(</span><span class="name">C</span><span class="operator">.</span><span class="name">shape</span><span class="punctuation">,</span> <span class="name">dtype</span><span class="operator">=</span><span class="name">np</span><span class="operator">.</span><span class="name">uint32</span><span class="punctuation">)</span>
    <span class="name">Z_</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">zeros</span><span class="punctuation">(</span><span class="name">C</span><span class="operator">.</span><span class="name">shape</span><span class="punctuation">,</span> <span class="name">dtype</span><span class="operator">=</span><span class="name">np</span><span class="operator">.</span><span class="name">complex64</span><span class="punctuation">)</span>
    <span class="name">Xi</span><span class="operator">.</span><span class="name">shape</span> <span class="operator">=</span> <span class="name">Yi</span><span class="operator">.</span><span class="name">shape</span> <span class="operator">=</span> <span class="name">C</span><span class="operator">.</span><span class="name">shape</span> <span class="operator">=</span> <span class="name">xn</span><span class="operator">*</span><span class="name">yn</span>

    <span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">zeros</span><span class="punctuation">(</span><span class="name">C</span><span class="operator">.</span><span class="name">shape</span><span class="punctuation">,</span> <span class="name">np</span><span class="operator">.</span><span class="name">complex64</span><span class="punctuation">)</span>
    <span class="keyword">for</span> <span class="name">i</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="name">itermax</span><span class="punctuation">):</span>
        <span class="keyword">if</span> <span class="operator word">not</span> <span class="name builtin">len</span><span class="punctuation">(</span><span class="name">Z</span><span class="punctuation">):</span> <span class="keyword">break</span>

        <span class="comment single"># Compute for relevant points only</span>
        <span class="name">np</span><span class="operator">.</span><span class="name">multiply</span><span class="punctuation">(</span><span class="name">Z</span><span class="punctuation">,</span> <span class="name">Z</span><span class="punctuation">,</span> <span class="name">Z</span><span class="punctuation">)</span>
        <span class="name">np</span><span class="operator">.</span><span class="name">add</span><span class="punctuation">(</span><span class="name">Z</span><span class="punctuation">,</span> <span class="name">C</span><span class="punctuation">,</span> <span class="name">Z</span><span class="punctuation">)</span>

        <span class="comment single"># Failed convergence</span>
        <span class="name">I</span> <span class="operator">=</span> <span class="name builtin">abs</span><span class="punctuation">(</span><span class="name">Z</span><span class="punctuation">)</span> <span class="operator">&gt;</span> <span class="name">horizon</span>
        <span class="name">N_</span><span class="punctuation">[</span><span class="name">Xi</span><span class="punctuation">[</span><span class="name">I</span><span class="punctuation">],</span> <span class="name">Yi</span><span class="punctuation">[</span><span class="name">I</span><span class="punctuation">]]</span> <span class="operator">=</span> <span class="name">i</span><span class="operator">+</span><span class="literal number integer">1</span>
        <span class="name">Z_</span><span class="punctuation">[</span><span class="name">Xi</span><span class="punctuation">[</span><span class="name">I</span><span class="punctuation">],</span> <span class="name">Yi</span><span class="punctuation">[</span><span class="name">I</span><span class="punctuation">]]</span> <span class="operator">=</span> <span class="name">Z</span><span class="punctuation">[</span><span class="name">I</span><span class="punctuation">]</span>

        <span class="comment single"># Keep going with those who have not diverged yet</span>
        <span class="name">np</span><span class="operator">.</span><span class="name">negative</span><span class="punctuation">(</span><span class="name">I</span><span class="punctuation">,</span><span class="name">I</span><span class="punctuation">)</span>
        <span class="name">Z</span> <span class="operator">=</span> <span class="name">Z</span><span class="punctuation">[</span><span class="name">I</span><span class="punctuation">]</span>
        <span class="name">Xi</span><span class="punctuation">,</span> <span class="name">Yi</span> <span class="operator">=</span> <span class="name">Xi</span><span class="punctuation">[</span><span class="name">I</span><span class="punctuation">],</span> <span class="name">Yi</span><span class="punctuation">[</span><span class="name">I</span><span class="punctuation">]</span>
        <span class="name">C</span> <span class="operator">=</span> <span class="name">C</span><span class="punctuation">[</span><span class="name">I</span><span class="punctuation">]</span>
    <span class="keyword">return</span> <span class="name">Z_</span><span class="operator">.</span><span class="name">T</span><span class="punctuation">,</span> <span class="name">N_</span><span class="operator">.</span><span class="name">T</span>
</pre>
<p>The benchmark gives us:</p>
<pre class="code pycon literal-block">
<span class="name"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;mandelbrot_numpy_2(xmin, xmax, ymin, ymax, xn, yn, maxiter)&quot;</span><span class="punctuation">,</span> <span class="name builtin">globals</span><span class="punctuation">())</span>
<span class="generic output">1 loops, best of 3: 510 msec per loop</span>
</pre>
</div>
<div class="section" id="visualization">
<h3><a class="toc-backref" href="#id6">Visualization</a></h3>
<p>In order to visualize our results, we could directly display the <code>N</code> array
using the matplotlib <code>imshow</code> command, but this would result in a &quot;banded&quot; image
that is a known consequence of the escape count algorithm that we've been
using. Such banding can be eliminated by using a fractional escape count. This
can be done by measuring how far the iterated point landed outside of the
escape cutoff. See the reference below about the renormalization of the escape
count. Here is a picture of the result where we use recount normalization,
and added a power normalized color map (gamma=0.3) as well as light shading.</p>
<div class="admonition legend">
<p class="first admonition-title"><strong>Figure 4.6</strong></p>
<p class="last">The Mandelbrot as rendered by matplotlib using recount normalization, power
normalized color map (gamma=0.3) and light shading.</p>
</div>
<div class="figure">
<img alt="data/mandelbrot.png" class="bordered" src="data/mandelbrot.png" style="width: 100%;" />
</div>
</div>
<div class="section" id="id10">
<h3><a class="toc-backref" href="#id6">Exercise</a></h3>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">You should look at the <a class="reference external" href="https://docs.scipy.org/doc/numpy/reference/generated/numpy.ufunc.reduceat.html">ufunc.reduceat</a> method that performs a (local) reduce with specified slices over a single axis.</p>
</div>
<p>We now want to measure the fractal dimension of the Mandelbrot set using the
<a class="reference external" href="https://en.wikipedia.org/wiki/Minkowski–Bouligand_dimension">Minkowski–Bouligand dimension</a>. To do that, we
need to do box-counting with a decreasing box size (see figure below). As you
can imagine, we cannot use pure Python because it would be way too slow. The goal of
the exercise is to write a function using NumPy that takes a two-dimensional
float array and returns the dimension. We'll consider values in the array to be
normalized (i.e. all values are between 0 and 1).</p>
<div class="admonition legend">
<p class="first admonition-title"><strong>Figure 4.7</strong></p>
<p class="last">The Minkowski–Bouligand dimension of the Great Britain coastlines is
approximately 1.24.</p>
</div>
<div class="figure">
<img alt="data/fractal-dimension.png" src="data/fractal-dimension.png" style="width: 100%;" />
</div>
</div>
<div class="section" id="id11">
<h3><a class="toc-backref" href="#id6">Sources</a></h3>
<ul class="simple">
<li><a class="reference external" href="code/mandelbrot.py">mandelbrot.py</a></li>
<li><a class="reference external" href="code/mandelbrot_python.py">mandelbrot_python.py</a></li>
<li><a class="reference external" href="code/mandelbrot_numpy_1.py">mandelbrot_numpy_1.py</a></li>
<li><a class="reference external" href="code/mandelbrot_numpy_2.py">mandelbrot_numpy_2.py</a></li>
<li><a class="reference external" href="code/fractal_dimension.py">fractal_dimension.py</a> (solution to the exercise)</li>
</ul>
</div>
<div class="section" id="id12">
<h3><a class="toc-backref" href="#id6">References</a></h3>
<ul class="simple">
<li><a class="reference external" href="https://www.ibm.com/developerworks/community/blogs/jfp/entry/How_To_Compute_Mandelbrodt_Set_Quickly?lang=en">How To Quickly Compute the Mandelbrot Set in Python</a>, Jean Francois Puget, 2015.</li>
<li><a class="reference external" href="https://www.ibm.com/developerworks/community/blogs/jfp/entry/My_Christmas_Gift?lang=en">My Christmas Gift: Mandelbrot Set Computation In Python</a>, Jean Francois Puget, 2015.</li>
<li><a class="reference external" href="https://thesamovar.wordpress.com/2009/03/22/fast-fractals-with-python-and-numpy/">Fast fractals with Python and NumPy</a>, Dan Goodman, 2009.</li>
<li><a class="reference external" href="http://linas.org/art-gallery/escape/escape.html">Renormalizing the Mandelbrot Escape</a>, Linas Vepstas, 1997.</li>
</ul>
</div>
</div>
<div class="section" id="spatial-vectorization">
<h2><a class="toc-backref" href="#id6">Spatial vectorization</a></h2>
<p>Spatial vectorization refers to a situation where elements share the same
computation but are in interaction with only a subgroup of other elements. This
was already the case for the game of life example, but in some situations
there is an added difficulty because the subgroup is dynamic and needs to be
updated at each iteration. This the case, for example, in particle systems where
particles interact mostly with local neighbours. This is also the case for
&quot;boids&quot; that simulate flocking behaviors.</p>
<div class="admonition legend">
<p class="first admonition-title"><strong>Figure 4.8</strong></p>
<p class="last">Flocking birds are an example of self-organization in biology.
Image by <a class="reference external" href="https://commons.wikimedia.org/wiki/File:Fugle,_ørnsø_073.jpg">Christoffer A Rasmussen</a>, 2012.</p>
</div>
<img alt="data/Fugle-cropped.jpg" class="bordered" src="data/Fugle-cropped.jpg" style="width: 100%;" />
<div class="section" id="boids">
<h3><a class="toc-backref" href="#id6">Boids</a></h3>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">Excerpt from the Wikipedia entry
<a class="reference external" href="https://en.wikipedia.org/wiki/Boids">Boids</a></p>
</div>
<p>Boids is an artificial life program, developed by Craig Reynolds in 1986, which
simulates the flocking behaviour of birds. The name &quot;boid&quot; corresponds to a
shortened version of &quot;bird-oid object&quot;, which refers to a bird-like object.</p>
<p>As with most artificial life simulations, Boids is an example of emergent
behavior; that is, the complexity of Boids arises from the interaction of
individual agents (the boids, in this case) adhering to a set of simple
rules. The rules applied in the simplest Boids world are as follows:</p>
<ul class="simple">
<li><strong>separation</strong>: steer to avoid crowding local flock-mates</li>
<li><strong>alignment</strong>: steer towards the average heading of local flock-mates</li>
<li><strong>cohesion</strong>: steer to move toward the average position (center of mass) of
local flock-mates</li>
</ul>
<div class="admonition legend">
<p class="first admonition-title"><strong>Figure 4.9</strong></p>
<p class="last">Boids are governed by a set of three local rules (separation, cohesion and
alignment) that serve as computing velocity and acceleration.</p>
</div>
<img alt="data/boids.png" src="data/boids.png" style="width: 100%;" />
</div>
<div class="section" id="id14">
<h3><a class="toc-backref" href="#id6">Python implementation</a></h3>
<p>Since each boid is an autonomous entity with several properties such as
position and velocity, it seems natural to start by writing a Boid class:</p>
<pre class="code python literal-block">
<span class="keyword namespace">import</span> <span class="name namespace">math</span>
<span class="keyword namespace">import</span> <span class="name namespace">random</span>
<span class="keyword namespace">from</span> <span class="name namespace">vec2</span> <span class="keyword namespace">import</span> <span class="name">vec2</span>

<span class="keyword">class</span> <span class="name class">Boid</span><span class="punctuation">:</span>
    <span class="keyword">def</span> <span class="name function magic">__init__</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="punctuation">,</span> <span class="name">x</span><span class="operator">=</span><span class="literal number integer">0</span><span class="punctuation">,</span> <span class="name">y</span><span class="operator">=</span><span class="literal number integer">0</span><span class="punctuation">):</span>
        <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">position</span> <span class="operator">=</span> <span class="name">vec2</span><span class="punctuation">(</span><span class="name">x</span><span class="punctuation">,</span> <span class="name">y</span><span class="punctuation">)</span>
        <span class="name">angle</span> <span class="operator">=</span> <span class="name">random</span><span class="operator">.</span><span class="name">uniform</span><span class="punctuation">(</span><span class="literal number integer">0</span><span class="punctuation">,</span> <span class="literal number integer">2</span><span class="operator">*</span><span class="name">math</span><span class="operator">.</span><span class="name">pi</span><span class="punctuation">)</span>
        <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">velocity</span> <span class="operator">=</span> <span class="name">vec2</span><span class="punctuation">(</span><span class="name">math</span><span class="operator">.</span><span class="name">cos</span><span class="punctuation">(</span><span class="name">angle</span><span class="punctuation">),</span> <span class="name">math</span><span class="operator">.</span><span class="name">sin</span><span class="punctuation">(</span><span class="name">angle</span><span class="punctuation">))</span>
        <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">acceleration</span> <span class="operator">=</span> <span class="name">vec2</span><span class="punctuation">(</span><span class="literal number integer">0</span><span class="punctuation">,</span> <span class="literal number integer">0</span><span class="punctuation">)</span>
</pre>
<p>The <code>vec2</code> object is a very simple class that handles all common vector
operations with 2 components. It will save us some writing in the main <code>Boid</code>
class. Note that there are some vector packages in the Python Package Index, but
that would be overkill for such a simple example.</p>
<p>Boid is a difficult case for regular Python because a boid has interaction with
local neighbours. However, and because boids are moving, to find such local
neighbours requires computing at each time step the distance to each and every
other boid in order to sort those which are in a given interaction radius. The
prototypical way of writing the three rules is thus something like:</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function">separation</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="punctuation">,</span> <span class="name">boids</span><span class="punctuation">):</span>
    <span class="name">count</span> <span class="operator">=</span> <span class="literal number integer">0</span>
    <span class="keyword">for</span> <span class="name">other</span> <span class="operator word">in</span> <span class="name">boids</span><span class="punctuation">:</span>
        <span class="name">d</span> <span class="operator">=</span> <span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">position</span> <span class="operator">-</span> <span class="name">other</span><span class="operator">.</span><span class="name">position</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">length</span><span class="punctuation">()</span>
        <span class="keyword">if</span> <span class="literal number integer">0</span> <span class="operator">&lt;</span> <span class="name">d</span> <span class="operator">&lt;</span> <span class="name">desired_separation</span><span class="punctuation">:</span>
            <span class="name">count</span> <span class="operator">+=</span> <span class="literal number integer">1</span>
            <span class="operator">...</span>
    <span class="keyword">if</span> <span class="name">count</span> <span class="operator">&gt;</span> <span class="literal number integer">0</span><span class="punctuation">:</span>
        <span class="operator">...</span>

 <span class="keyword">def</span> <span class="name function">alignment</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="punctuation">,</span> <span class="name">boids</span><span class="punctuation">):</span> <span class="operator">...</span>
 <span class="keyword">def</span> <span class="name function">cohesion</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="punctuation">,</span> <span class="name">boids</span><span class="punctuation">):</span> <span class="operator">...</span>
</pre>
<p>Full sources are given in the references section below, it would be too long to
describe it here and there is no real difficulty.</p>
<p>To complete the picture, we can also create a <code>Flock</code> object:</p>
<pre class="code python literal-block">
<span class="keyword">class</span> <span class="name class">Flock</span><span class="punctuation">:</span>
    <span class="keyword">def</span> <span class="name function magic">__init__</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="punctuation">,</span> <span class="name">count</span><span class="operator">=</span><span class="literal number integer">150</span><span class="punctuation">):</span>
        <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">boids</span> <span class="operator">=</span> <span class="punctuation">[]</span>
        <span class="keyword">for</span> <span class="name">i</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="name">count</span><span class="punctuation">):</span>
            <span class="name">boid</span> <span class="operator">=</span> <span class="name">Boid</span><span class="punctuation">()</span>
            <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">boids</span><span class="operator">.</span><span class="name">append</span><span class="punctuation">(</span><span class="name">boid</span><span class="punctuation">)</span>

    <span class="keyword">def</span> <span class="name function">run</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="punctuation">):</span>
        <span class="keyword">for</span> <span class="name">boid</span> <span class="operator word">in</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">boids</span><span class="punctuation">:</span>
            <span class="name">boid</span><span class="operator">.</span><span class="name">run</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">boids</span><span class="punctuation">)</span>
</pre>
<p>Using this approach, we can have up to 50 boids until the computation
time becomes too slow for a smooth animation. As you may have guessed,
we can do much better using NumPy, but let me first point out the main
problem with this Python implementation. If you look at the code, you
will certainly notice there is a lot of redundancy. More precisely, we
do not exploit the fact that the Euclidean distance is reflexive, that
is, <span class="formula">|<i>x</i> − <i>y</i>| = |<i>y</i> − <i>x</i>|</span>. In this naive Python implementation, each
rule (function) computes <span class="formula"><i>n</i><sup>2</sup></span> distances while
<span class="formula"><span class="fraction"><span class="ignored">(</span><span class="numerator"><i>n</i><sup>2</sup></span><span class="ignored">)/(</span><span class="denominator">2</span><span class="ignored">)</span></span></span> would be sufficient if properly
cached. Furthermore, each rule re-computes every distance without
caching the result for the other functions. In the end, we are
computing <span class="formula">3<i>n</i><sup>2</sup></span> distances instead of <span class="formula"><span class="fraction"><span class="ignored">(</span><span class="numerator"><i>n</i><sup>2</sup></span><span class="ignored">)/(</span><span class="denominator">2</span><span class="ignored">)</span></span></span>.</p>
</div>
<div class="section" id="id15">
<h3><a class="toc-backref" href="#id6">NumPy implementation</a></h3>
<p>As you might expect, the NumPy implementation takes a different approach and
we'll gather all our boids into a <code>position</code> array and a <code>velocity</code> array:</p>
<pre class="code python literal-block">
<span class="name">n</span> <span class="operator">=</span> <span class="literal number integer">500</span>
<span class="name">velocity</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">zeros</span><span class="punctuation">((</span><span class="name">n</span><span class="punctuation">,</span> <span class="literal number integer">2</span><span class="punctuation">),</span> <span class="name">dtype</span><span class="operator">=</span><span class="name">np</span><span class="operator">.</span><span class="name">float32</span><span class="punctuation">)</span>
<span class="name">position</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">zeros</span><span class="punctuation">((</span><span class="name">n</span><span class="punctuation">,</span> <span class="literal number integer">2</span><span class="punctuation">),</span> <span class="name">dtype</span><span class="operator">=</span><span class="name">np</span><span class="operator">.</span><span class="name">float32</span><span class="punctuation">)</span>
</pre>
<p>The first step is to compute the local neighborhood for all boids, and for
this we need to compute all paired distances:</p>
<pre class="code python literal-block">
<span class="name">dx</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">subtract</span><span class="operator">.</span><span class="name">outer</span><span class="punctuation">(</span><span class="name">position</span><span class="punctuation">[:,</span> <span class="literal number integer">0</span><span class="punctuation">],</span> <span class="name">position</span><span class="punctuation">[:,</span> <span class="literal number integer">0</span><span class="punctuation">])</span>
<span class="name">dy</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">subtract</span><span class="operator">.</span><span class="name">outer</span><span class="punctuation">(</span><span class="name">position</span><span class="punctuation">[:,</span> <span class="literal number integer">1</span><span class="punctuation">],</span> <span class="name">position</span><span class="punctuation">[:,</span> <span class="literal number integer">1</span><span class="punctuation">])</span>
<span class="name">distance</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">hypot</span><span class="punctuation">(</span><span class="name">dx</span><span class="punctuation">,</span> <span class="name">dy</span><span class="punctuation">)</span>
</pre>
<p>We could have used the scipy <a class="reference external" href="https://docs.scipy.org/doc/scipy/reference/generated/scipy.spatial.distance.cdist.html">cdist</a>
but we'll need the <code>dx</code> and <code>dy</code> arrays later. Once those have been computed,
it is faster to use the <a class="reference external" href="https://docs.scipy.org/doc/numpy/reference/generated/numpy.hypot.html">hypot</a>
method. Note that distance shape is <code>(n, n)</code> and each line relates to one boid,
i.e. each line gives the distance to all other boids (including self).</p>
<p>From theses distances, we can now compute the local neighborhood for
each of the three rules, taking advantage of the fact that we can mix
them together. We can actually compute a mask for distances that are
strictly positive (i.e. have no self-interaction) and multiply it with
other distance masks.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">If we suppose that boids cannot occupy the same position, how can you
compute <code>mask_0</code> more efficiently?</p>
</div>
<pre class="code python literal-block">
<span class="name">mask_0</span> <span class="operator">=</span> <span class="punctuation">(</span><span class="name">distance</span> <span class="operator">&gt;</span> <span class="literal number integer">0</span><span class="punctuation">)</span>
<span class="name">mask_1</span> <span class="operator">=</span> <span class="punctuation">(</span><span class="name">distance</span> <span class="operator">&lt;</span> <span class="literal number integer">25</span><span class="punctuation">)</span>
<span class="name">mask_2</span> <span class="operator">=</span> <span class="punctuation">(</span><span class="name">distance</span> <span class="operator">&lt;</span> <span class="literal number integer">50</span><span class="punctuation">)</span>
<span class="name">mask_1</span> <span class="operator">*=</span> <span class="name">mask_0</span>
<span class="name">mask_2</span> <span class="operator">*=</span> <span class="name">mask_0</span>
<span class="name">mask_3</span> <span class="operator">=</span> <span class="name">mask_2</span>
</pre>
<p>Then, we compute the number of neighbours within the given radius and we ensure
it is at least 1 to avoid division by zero.</p>
<pre class="code python literal-block">
<span class="name">mask_1_count</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">maximum</span><span class="punctuation">(</span><span class="name">mask_1</span><span class="operator">.</span><span class="name">sum</span><span class="punctuation">(</span><span class="name">axis</span><span class="operator">=</span><span class="literal number integer">1</span><span class="punctuation">),</span> <span class="literal number integer">1</span><span class="punctuation">)</span>
<span class="name">mask_2_count</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">maximum</span><span class="punctuation">(</span><span class="name">mask_2</span><span class="operator">.</span><span class="name">sum</span><span class="punctuation">(</span><span class="name">axis</span><span class="operator">=</span><span class="literal number integer">1</span><span class="punctuation">),</span> <span class="literal number integer">1</span><span class="punctuation">)</span>
<span class="name">mask_3_count</span> <span class="operator">=</span> <span class="name">mask_2_count</span>
</pre>
<p>We're ready to write our three rules:</p>
<p><strong>Alignment</strong></p>
<pre class="code python literal-block">
<span class="comment single"># Compute the average velocity of local neighbours</span>
<span class="name">target</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">dot</span><span class="punctuation">(</span><span class="name">mask</span><span class="punctuation">,</span> <span class="name">velocity</span><span class="punctuation">)</span><span class="operator">/</span><span class="name">count</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="name">n</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">)</span>

<span class="comment single"># Normalize the result</span>
<span class="name">norm</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">sqrt</span><span class="punctuation">((</span><span class="name">target</span><span class="operator">*</span><span class="name">target</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">sum</span><span class="punctuation">(</span><span class="name">axis</span><span class="operator">=</span><span class="literal number integer">1</span><span class="punctuation">))</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="name">n</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">)</span>
<span class="name">target</span> <span class="operator">*=</span> <span class="name">np</span><span class="operator">.</span><span class="name">divide</span><span class="punctuation">(</span><span class="name">target</span><span class="punctuation">,</span> <span class="name">norm</span><span class="punctuation">,</span> <span class="name">out</span><span class="operator">=</span><span class="name">target</span><span class="punctuation">,</span> <span class="name">where</span><span class="operator">=</span><span class="name">norm</span> <span class="operator">!=</span> <span class="literal number integer">0</span><span class="punctuation">)</span>

<span class="comment single"># Alignment at constant speed</span>
<span class="name">target</span> <span class="operator">*=</span> <span class="name">max_velocity</span>

<span class="comment single"># Compute the resulting steering</span>
<span class="name">alignment</span> <span class="operator">=</span> <span class="name">target</span> <span class="operator">-</span> <span class="name">velocity</span>
</pre>
<p><strong>Cohesion</strong></p>
<pre class="code python literal-block">
<span class="comment single"># Compute the gravity center of local neighbours</span>
<span class="name">center</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">dot</span><span class="punctuation">(</span><span class="name">mask</span><span class="punctuation">,</span> <span class="name">position</span><span class="punctuation">)</span><span class="operator">/</span><span class="name">count</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="name">n</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">)</span>

<span class="comment single"># Compute direction toward the center</span>
<span class="name">target</span> <span class="operator">=</span> <span class="name">center</span> <span class="operator">-</span> <span class="name">position</span>

<span class="comment single"># Normalize the result</span>
<span class="name">norm</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">sqrt</span><span class="punctuation">((</span><span class="name">target</span><span class="operator">*</span><span class="name">target</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">sum</span><span class="punctuation">(</span><span class="name">axis</span><span class="operator">=</span><span class="literal number integer">1</span><span class="punctuation">))</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="name">n</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">)</span>
<span class="name">target</span> <span class="operator">*=</span> <span class="name">np</span><span class="operator">.</span><span class="name">divide</span><span class="punctuation">(</span><span class="name">target</span><span class="punctuation">,</span> <span class="name">norm</span><span class="punctuation">,</span> <span class="name">out</span><span class="operator">=</span><span class="name">target</span><span class="punctuation">,</span> <span class="name">where</span><span class="operator">=</span><span class="name">norm</span> <span class="operator">!=</span> <span class="literal number integer">0</span><span class="punctuation">)</span>

<span class="comment single"># Cohesion at constant speed (max_velocity)</span>
<span class="name">target</span> <span class="operator">*=</span> <span class="name">max_velocity</span>

<span class="comment single"># Compute the resulting steering</span>
<span class="name">cohesion</span> <span class="operator">=</span> <span class="name">target</span> <span class="operator">-</span> <span class="name">velocity</span>
</pre>
<p><strong>Separation</strong></p>
<pre class="code python literal-block">
<span class="comment single"># Compute the repulsion force from local neighbours</span>
<span class="name">repulsion</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">dstack</span><span class="punctuation">((</span><span class="name">dx</span><span class="punctuation">,</span> <span class="name">dy</span><span class="punctuation">))</span>

<span class="comment single"># Force is inversely proportional to the distance</span>
<span class="name">repulsion</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">divide</span><span class="punctuation">(</span><span class="name">repulsion</span><span class="punctuation">,</span> <span class="name">distance</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="name">n</span><span class="punctuation">,</span> <span class="name">n</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">)</span><span class="operator">**</span><span class="literal number integer">2</span><span class="punctuation">,</span> <span class="name">out</span><span class="operator">=</span><span class="name">repulsion</span><span class="punctuation">,</span>
                      <span class="name">where</span><span class="operator">=</span><span class="name">distance</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="name">n</span><span class="punctuation">,</span> <span class="name">n</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">)</span> <span class="operator">!=</span> <span class="literal number integer">0</span><span class="punctuation">)</span>

<span class="comment single"># Compute direction away from others</span>
<span class="name">target</span> <span class="operator">=</span> <span class="punctuation">(</span><span class="name">repulsion</span><span class="operator">*</span><span class="name">mask</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="name">n</span><span class="punctuation">,</span> <span class="name">n</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">))</span><span class="operator">.</span><span class="name">sum</span><span class="punctuation">(</span><span class="name">axis</span><span class="operator">=</span><span class="literal number integer">1</span><span class="punctuation">)</span><span class="operator">/</span><span class="name">count</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="name">n</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">)</span>

<span class="comment single"># Normalize the result</span>
<span class="name">norm</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">sqrt</span><span class="punctuation">((</span><span class="name">target</span><span class="operator">*</span><span class="name">target</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">sum</span><span class="punctuation">(</span><span class="name">axis</span><span class="operator">=</span><span class="literal number integer">1</span><span class="punctuation">))</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="name">n</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">)</span>
<span class="name">target</span> <span class="operator">*=</span> <span class="name">np</span><span class="operator">.</span><span class="name">divide</span><span class="punctuation">(</span><span class="name">target</span><span class="punctuation">,</span> <span class="name">norm</span><span class="punctuation">,</span> <span class="name">out</span><span class="operator">=</span><span class="name">target</span><span class="punctuation">,</span> <span class="name">where</span><span class="operator">=</span><span class="name">norm</span> <span class="operator">!=</span> <span class="literal number integer">0</span><span class="punctuation">)</span>

<span class="comment single"># Separation at constant speed (max_velocity)</span>
<span class="name">target</span> <span class="operator">*=</span> <span class="name">max_velocity</span>

<span class="comment single"># Compute the resulting steering</span>
<span class="name">separation</span> <span class="operator">=</span> <span class="name">target</span> <span class="operator">-</span> <span class="name">velocity</span>
</pre>
<p>All three resulting steerings (separation, alignment &amp; cohesion) need to
be limited in magnitude. We leave this as an exercise for the reader. Combination
of these rules is straightforward as well as the resulting update of
velocity and position:</p>
<pre class="code python literal-block">
<span class="name">acceleration</span> <span class="operator">=</span> <span class="literal number float">1.5</span> <span class="operator">*</span> <span class="name">separation</span> <span class="operator">+</span> <span class="name">alignment</span> <span class="operator">+</span> <span class="name">cohesion</span>
<span class="name">velocity</span> <span class="operator">+=</span> <span class="name">acceleration</span>
<span class="name">position</span> <span class="operator">+=</span> <span class="name">velocity</span>
</pre>
<p>We finally visualize the result using a custom oriented scatter plot.</p>
<div class="admonition legend">
<p class="first admonition-title"><strong>Figure 4.10</strong></p>
<p class="last">Boids is an artificial life program, developed by Craig Reynolds in 1986,
which simulates the flocking behaviour of birds.</p>
</div>
<video width="100%" class="bordered" controls>
<source src="data/boids.mp4" type="video/mp4">
Your browser does not support the video tag. </video></div>
<div class="section" id="id16">
<h3><a class="toc-backref" href="#id6">Exercise</a></h3>
<p>We are now ready to visualize our boids. The easiest way is to use the
matplotlib animation function and a scatter plot. Unfortunately, scatters
cannot be individually oriented and we need to make our own objects using a
matplotlib <code>PathCollection</code>. A simple triangle path can be defined as:</p>
<pre class="code literal-block">
v= np.array([(-0.25, -0.25),
             ( 0.00,  0.50),
             ( 0.25, -0.25),
             ( 0.00,  0.00)])
c = np.array([Path.MOVETO,
              Path.LINETO,
              Path.LINETO,
              Path.CLOSEPOLY])
</pre>
<p>This path can be repeated several times inside an array and each triangle can
be made independent.</p>
<pre class="code literal-block">
n = 500
vertices = np.tile(v.reshape(-1), n).reshape(n, len(v), 2)
codes = np.tile(c.reshape(-1), n)
</pre>
<p>We now have a <code>(n,4,2)</code> array for vertices and a <code>(n,4)</code> array for codes
representing <code>n</code> boids. We are interested in manipulating the vertices array to
reflect the translation, scaling and rotation of each of the <code>n</code> boids.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">Rotate is really tricky.</p>
</div>
<p>How would you write the <code>translate</code>, <code>scale</code> and <code>rotate</code> functions ?</p>
</div>
<div class="section" id="id17">
<h3><a class="toc-backref" href="#id6">Sources</a></h3>
<ul class="simple">
<li><a class="reference external" href="code/boid_python.py">boid_python.py</a></li>
<li><a class="reference external" href="code/boid_numpy.py">boid_numpy.py</a> (solution to the exercise)</li>
</ul>
</div>
<div class="section" id="id18">
<h3><a class="toc-backref" href="#id6">References</a></h3>
<ul class="simple">
<li><a class="reference external" href="https://processing.org/examples/flocking.html">Flocking</a>, Daniel Shiffman, 2010.</li>
<li><a class="reference external" href="http://www.red3d.com/cwr/boids/">Flocks, herds and schools: A distributed behavioral model</a>, Craig Reynolds, SIGGRAPH, 1987</li>
</ul>
</div>
</div>
<div class="section" id="id19">
<h2><a class="toc-backref" href="#id6">Conclusion</a></h2>
<p>We've seen through these examples three forms of code vectorization:</p>
<ul class="simple">
<li>Uniform vectorization where elements share the same computation
unconditionally and for the same duration.</li>
<li>Temporal vectorization where elements share the same computation but
necessitate a different number of iterations.</li>
<li>Spatial vectorization where elements share the same computation but on
dynamic spatial arguments.</li>
</ul>
<p>And there are probably many more forms of such direct code vectorization. As
explained before, this kind of vectorization is one of the most simple even
though we've seen it can be really tricky to implement and requires some
experience, some help or both. For example, the solution to the boids exercise
was provided by <a class="reference external" href="http://stackoverflow.com/users/3293881/divakar">Divakar</a> on
<a class="reference external" href="http://stackoverflow.com/questions/40822983/multiple-individual-2d-rotation-at-once">stack overflow</a> after having explained my problem.</p>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<!-- Title:   From Python to Numpy -->
<!-- Author:  Nicolas P. Rougier -->
<!-- Date:    January 2017 -->
<!-- License: Creative Commons Share-Alike Non-Commercial International 4.0 -->
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
</div>
</div>
<div class="section" id="problem-vectorization">
<h1><a class="toc-backref" href="#table-of-contents">Problem vectorization</a></h1>
<div class="contents local topic" id="id20">
<p class="topic-title first"><strong>Contents</strong></p>
<ul class="simple">
<li><a class="reference internal" href="#id21" id="id146">Introduction</a></li>
<li><a class="reference internal" href="#path-finding" id="id147">Path finding</a><ul>
<li><a class="reference internal" href="#building-a-maze" id="id148">Building a maze</a></li>
<li><a class="reference internal" href="#breadth-first" id="id149">Breadth-first</a></li>
<li><a class="reference internal" href="#bellman-ford-method" id="id150">Bellman-Ford method</a></li>
<li><a class="reference internal" href="#id22" id="id151">Sources</a></li>
<li><a class="reference internal" href="#id23" id="id152">References</a></li>
</ul>
</li>
<li><a class="reference internal" href="#fluid-dynamics" id="id153">Fluid Dynamics</a><ul>
<li><a class="reference internal" href="#lagrangian-vs-eulerian-method" id="id154">Lagrangian vs Eulerian method</a></li>
<li><a class="reference internal" href="#id24" id="id155">NumPy implementation</a></li>
<li><a class="reference internal" href="#id25" id="id156">Sources</a></li>
<li><a class="reference internal" href="#id26" id="id157">References</a></li>
</ul>
</li>
<li><a class="reference internal" href="#blue-noise-sampling" id="id158">Blue noise sampling</a><ul>
<li><a class="reference internal" href="#dart-method" id="id159">DART method</a></li>
<li><a class="reference internal" href="#bridson-method" id="id160">Bridson method</a></li>
<li><a class="reference internal" href="#id27" id="id161">Sources</a></li>
<li><a class="reference internal" href="#id28" id="id162">References</a></li>
</ul>
</li>
<li><a class="reference internal" href="#id29" id="id163">Conclusion</a></li>
</ul>
</div>
<div class="section" id="id21">
<h2><a class="toc-backref" href="#id20">Introduction</a></h2>
<p>Problem vectorization is much harder than code vectorization because it means
that you fundamentally have to rethink your problem in order to make it
vectorizable. Most of the time this means you have to use a different algorithm
to solve your problem or even worse... to invent a new one. The difficulty is thus
to think out-of-the-box.</p>
<p>To illustrate this, let's consider a simple problem where given two vectors <code>X</code> and
<code>Y</code>, we want to compute the sum of <code>X[i]*Y[j]</code> for all pairs of indices <code>i</code>,
<code>j</code>. One simple and obvious solution is to write:</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function">compute_python</span><span class="punctuation">(</span><span class="name">X</span><span class="punctuation">,</span> <span class="name">Y</span><span class="punctuation">):</span>
    <span class="name">result</span> <span class="operator">=</span> <span class="literal number integer">0</span>
    <span class="keyword">for</span> <span class="name">i</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="name builtin">len</span><span class="punctuation">(</span><span class="name">X</span><span class="punctuation">)):</span>
        <span class="keyword">for</span> <span class="name">j</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="name builtin">len</span><span class="punctuation">(</span><span class="name">Y</span><span class="punctuation">)):</span>
            <span class="name">result</span> <span class="operator">+=</span> <span class="name">X</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">]</span> <span class="operator">*</span> <span class="name">Y</span><span class="punctuation">[</span><span class="name">j</span><span class="punctuation">]</span>
    <span class="keyword">return</span> <span class="name">result</span>
</pre>
<p>However, this first and naïve implementation requires two loops and we already
know it will be slow:</p>
<pre class="code python literal-block">
<span class="operator">&gt;&gt;&gt;</span> <span class="name">X</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">arange</span><span class="punctuation">(</span><span class="literal number integer">1000</span><span class="punctuation">)</span>
<span class="operator">&gt;&gt;&gt;</span> <span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;compute_python(X,X)&quot;</span><span class="punctuation">)</span>
<span class="literal number integer">1</span> <span class="name">loops</span><span class="punctuation">,</span> <span class="name">best</span> <span class="name">of</span> <span class="literal number integer">3</span><span class="punctuation">:</span> <span class="literal number float">0.274481</span> <span class="name">sec</span> <span class="name">per</span> <span class="name">loop</span>
</pre>
<p>How to vectorize the problem then? If you remember your linear algebra course,
you may have identified the expression <code>X[i] * Y[j]</code> to be very similar to a
matrix product expression. So maybe we could benefit from some NumPy
speedup. One wrong solution would be to write:</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function">compute_numpy_wrong</span><span class="punctuation">(</span><span class="name">X</span><span class="punctuation">,</span> <span class="name">Y</span><span class="punctuation">):</span>
    <span class="keyword">return</span> <span class="punctuation">(</span><span class="name">X</span><span class="operator">*</span><span class="name">Y</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">sum</span><span class="punctuation">()</span>
</pre>
<p>This is wrong because the <code>X*Y</code> expression will actually compute a new vector
<code>Z</code> such that <code>Z[i] = X[i] * Y[i]</code> and this is not what we want. Instead, we
can exploit NumPy broadcasting by first reshaping the two vectors and then
multiply them:</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function">compute_numpy</span><span class="punctuation">(</span><span class="name">X</span><span class="punctuation">,</span> <span class="name">Y</span><span class="punctuation">):</span>
    <span class="name">Z</span> <span class="operator">=</span> <span class="name">X</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="name builtin">len</span><span class="punctuation">(</span><span class="name">X</span><span class="punctuation">),</span><span class="literal number integer">1</span><span class="punctuation">)</span> <span class="operator">*</span> <span class="name">Y</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="name builtin">len</span><span class="punctuation">(</span><span class="name">Y</span><span class="punctuation">))</span>
    <span class="keyword">return</span> <span class="name">Z</span><span class="operator">.</span><span class="name">sum</span><span class="punctuation">()</span>
</pre>
<p>Here we have <code>Z[i,j] == X[i,0]*Y[0,j]</code> and if we take the sum over each elements of
<code>Z</code>, we get the expected result. Let's see how much speedup we gain in the
process:</p>
<pre class="code python literal-block">
<span class="operator">&gt;&gt;&gt;</span> <span class="name">X</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">arange</span><span class="punctuation">(</span><span class="literal number integer">1000</span><span class="punctuation">)</span>
<span class="operator">&gt;&gt;&gt;</span> <span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;compute_numpy(X,X)&quot;</span><span class="punctuation">)</span>
<span class="literal number integer">10</span> <span class="name">loops</span><span class="punctuation">,</span> <span class="name">best</span> <span class="name">of</span> <span class="literal number integer">3</span><span class="punctuation">:</span> <span class="literal number float">0.00157926</span> <span class="name">sec</span> <span class="name">per</span> <span class="name">loop</span>
</pre>
<p>This is better, we gained a factor of ~150. But we can do much better.</p>
<p>If you look again and more closely at the pure Python version, you can see that
the inner loop is using <code>X[i]</code> that does not depend on the <code>j</code> index, meaning
it can be removed from the inner loop. Code can be rewritten as:</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function">compute_numpy_better_1</span><span class="punctuation">(</span><span class="name">X</span><span class="punctuation">,</span> <span class="name">Y</span><span class="punctuation">):</span>
    <span class="name">result</span> <span class="operator">=</span> <span class="literal number integer">0</span>
    <span class="keyword">for</span> <span class="name">i</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="name builtin">len</span><span class="punctuation">(</span><span class="name">X</span><span class="punctuation">)):</span>
        <span class="name">Ysum</span> <span class="operator">=</span> <span class="literal number integer">0</span>
        <span class="keyword">for</span> <span class="name">j</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="name builtin">len</span><span class="punctuation">(</span><span class="name">Y</span><span class="punctuation">)):</span>
            <span class="name">Ysum</span> <span class="operator">+=</span> <span class="name">Y</span><span class="punctuation">[</span><span class="name">j</span><span class="punctuation">]</span>
        <span class="name">result</span> <span class="operator">+=</span> <span class="name">X</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">]</span><span class="operator">*</span><span class="name">Ysum</span>
    <span class="keyword">return</span> <span class="name">result</span>
</pre>
<p>But since the inner loop does not depend on the <code>i</code> index, we might as well
compute it only once:</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function">compute_numpy_better_2</span><span class="punctuation">(</span><span class="name">X</span><span class="punctuation">,</span> <span class="name">Y</span><span class="punctuation">):</span>
    <span class="name">result</span> <span class="operator">=</span> <span class="literal number integer">0</span>
    <span class="name">Ysum</span> <span class="operator">=</span> <span class="literal number integer">0</span>
    <span class="keyword">for</span> <span class="name">j</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="name builtin">len</span><span class="punctuation">(</span><span class="name">Y</span><span class="punctuation">)):</span>
        <span class="name">Ysum</span> <span class="operator">+=</span> <span class="name">Y</span><span class="punctuation">[</span><span class="name">j</span><span class="punctuation">]</span>
    <span class="keyword">for</span> <span class="name">i</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="name builtin">len</span><span class="punctuation">(</span><span class="name">X</span><span class="punctuation">)):</span>
        <span class="name">result</span> <span class="operator">+=</span> <span class="name">X</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">]</span><span class="operator">*</span><span class="name">Ysum</span>
    <span class="keyword">return</span> <span class="name">result</span>
</pre>
<p>Not so bad, we have removed the inner loop, meaning with transform a
<span class="formula"><i>O</i>(<i>n</i><sup>2</sup>)</span> complexity into <span class="formula"><i>O</i>(<i>n</i>)</span> complexity. Using the same
approach, we can now write:</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function">compute_numpy_better_3</span><span class="punctuation">(</span><span class="name">x</span><span class="punctuation">,</span> <span class="name">y</span><span class="punctuation">):</span>
    <span class="name">Ysum</span> <span class="operator">=</span> <span class="literal number integer">0</span>
    <span class="keyword">for</span> <span class="name">j</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="name builtin">len</span><span class="punctuation">(</span><span class="name">Y</span><span class="punctuation">)):</span>
        <span class="name">Ysum</span> <span class="operator">+=</span> <span class="name">Y</span><span class="punctuation">[</span><span class="name">j</span><span class="punctuation">]</span>
    <span class="name">Xsum</span> <span class="operator">=</span> <span class="literal number integer">0</span>
    <span class="keyword">for</span> <span class="name">i</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="name builtin">len</span><span class="punctuation">(</span><span class="name">X</span><span class="punctuation">)):</span>
        <span class="name">Xsum</span> <span class="operator">+=</span> <span class="name">X</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">]</span>
    <span class="keyword">return</span> <span class="name">Xsum</span><span class="operator">*</span><span class="name">Ysum</span>
</pre>
<p>Finally, having realized we only need the product of the sum over <code>X</code> and <code>Y</code>
respectively, we can benefit from the <code>np.sum</code> function and write:</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function">compute_numpy_better</span><span class="punctuation">(</span><span class="name">x</span><span class="punctuation">,</span> <span class="name">y</span><span class="punctuation">):</span>
    <span class="keyword">return</span> <span class="name">np</span><span class="operator">.</span><span class="name">sum</span><span class="punctuation">(</span><span class="name">y</span><span class="punctuation">)</span> <span class="operator">*</span> <span class="name">np</span><span class="operator">.</span><span class="name">sum</span><span class="punctuation">(</span><span class="name">x</span><span class="punctuation">)</span>
</pre>
<p>It is shorter, clearer and much, much faster:</p>
<pre class="code python literal-block">
<span class="operator">&gt;&gt;&gt;</span> <span class="name">X</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">arange</span><span class="punctuation">(</span><span class="literal number integer">1000</span><span class="punctuation">)</span>
<span class="operator">&gt;&gt;&gt;</span> <span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;compute_numpy_better(X,X)&quot;</span><span class="punctuation">)</span>
<span class="literal number integer">1000</span> <span class="name">loops</span><span class="punctuation">,</span> <span class="name">best</span> <span class="name">of</span> <span class="literal number integer">3</span><span class="punctuation">:</span> <span class="literal number float">3.97208e-06</span> <span class="name">sec</span> <span class="name">per</span> <span class="name">loop</span>
</pre>
<p>We have indeed reformulated our problem, taking advantage of the fact that
<span class="formula"><span class="limits"><span class="limit">∑</span></span><sub><i>ij</i></sub><i>X</i><sub><i>i</i></sub><i>Y</i><sub><i>j</i></sub> = <span class="limits"><span class="limit">∑</span></span><sub><i>i</i></sub><i>X</i><sub><i>i</i></sub><span class="limits"><span class="limit">∑</span></span><sub><i>j</i></sub><i>Y</i><sub><i>j</i></sub></span> and we've learned in the
meantime that there are two kinds of vectorization: code vectorization and
problem vectorization. The latter is the most difficult but the most
important because this is where you can expect huge gains in speed. In this
simple example, we gain a factor of 150 with code vectorization but we gained a
factor of 70,000 with problem vectorization, just by writing our problem
differently (even though you cannot expect such a huge speedup in all
situations). However, code vectorization remains an important factor, and if we
rewrite the last solution the Python way, the improvement is good but not as much as
in the NumPy version:</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function">compute_python_better</span><span class="punctuation">(</span><span class="name">x</span><span class="punctuation">,</span> <span class="name">y</span><span class="punctuation">):</span>
    <span class="keyword">return</span> <span class="name builtin">sum</span><span class="punctuation">(</span><span class="name">x</span><span class="punctuation">)</span><span class="operator">*</span><span class="name builtin">sum</span><span class="punctuation">(</span><span class="name">y</span><span class="punctuation">)</span>
</pre>
<p>This new Python version is much faster than the previous Python version, but
still, it is 50 times slower than the NumPy version:</p>
<pre class="code python literal-block">
<span class="operator">&gt;&gt;&gt;</span> <span class="name">X</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">arange</span><span class="punctuation">(</span><span class="literal number integer">1000</span><span class="punctuation">)</span>
<span class="operator">&gt;&gt;&gt;</span> <span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;compute_python_better(X,X)&quot;</span><span class="punctuation">)</span>
<span class="literal number integer">1000</span> <span class="name">loops</span><span class="punctuation">,</span> <span class="name">best</span> <span class="name">of</span> <span class="literal number integer">3</span><span class="punctuation">:</span> <span class="literal number float">0.000155677</span> <span class="name">sec</span> <span class="name">per</span> <span class="name">loop</span>
</pre>
</div>
<div class="section" id="path-finding">
<h2><a class="toc-backref" href="#id20">Path finding</a></h2>
<p>Path finding is all about finding the shortest path in a graph. This can be
split in two distinct problems: to find a path between two nodes in a graph and
to find the shortest path. We'll illustrate this through path finding in a
maze. The first task is thus to build a maze.</p>
<div class="admonition legend">
<p class="first admonition-title"><strong>Figure 5.1</strong></p>
<p class="last">A hedge maze at Longleat stately home in England.
Image by <a class="reference external" href="https://commons.wikimedia.org/wiki/File:Longleat_maze.jpg">Prince Rurik</a>, 2005.</p>
</div>
<img alt="data//Longleat-maze-cropped.jpg" class="bordered" src="data//Longleat-maze-cropped.jpg" style="width: 100%;" />
<div class="section" id="building-a-maze">
<h3><a class="toc-backref" href="#id20">Building a maze</a></h3>
<p>There exist <a class="reference external" href="https://en.wikipedia.org/wiki/Maze_generation_algorithm">many maze generation algorithms</a> but I tend to
prefer the one I've been using for several years but whose origin is unknown to
me. I've added the code in the cited wikipedia entry. Feel free to complete it
if you know the original author. This algorithm works by creating <code>n</code> (density)
islands of length <code>p</code> (complexity). An island is created by choosing a random
starting point with odd coordinates, then a random direction is chosen. If the
cell two steps in a given direction is free, then a wall is added at both one step
and two steps in this direction. The process is iterated for <code>n</code> steps for this
island. <code>p</code> islands are created. <code>n</code> and <code>p</code> are expressed as <code>float</code> to adapt them to
the size of the maze. With a low complexity, islands are very small and the
maze is easy to solve. With low density, the maze has more &quot;big empty rooms&quot;.</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function">build_maze</span><span class="punctuation">(</span><span class="name">shape</span><span class="operator">=</span><span class="punctuation">(</span><span class="literal number integer">65</span><span class="punctuation">,</span> <span class="literal number integer">65</span><span class="punctuation">),</span> <span class="name">complexity</span><span class="operator">=</span><span class="literal number float">0.75</span><span class="punctuation">,</span> <span class="name">density</span><span class="operator">=</span><span class="literal number float">0.50</span><span class="punctuation">):</span>
    <span class="comment single"># Only odd shapes</span>
    <span class="name">shape</span> <span class="operator">=</span> <span class="punctuation">((</span><span class="name">shape</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">]</span><span class="operator">//</span><span class="literal number integer">2</span><span class="punctuation">)</span><span class="operator">*</span><span class="literal number integer">2</span><span class="operator">+</span><span class="literal number integer">1</span><span class="punctuation">,</span> <span class="punctuation">(</span><span class="name">shape</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">]</span><span class="operator">//</span><span class="literal number integer">2</span><span class="punctuation">)</span><span class="operator">*</span><span class="literal number integer">2</span><span class="operator">+</span><span class="literal number integer">1</span><span class="punctuation">)</span>

    <span class="comment single"># Adjust complexity and density relatively to maze size</span>
    <span class="name">n_complexity</span> <span class="operator">=</span> <span class="name builtin">int</span><span class="punctuation">(</span><span class="name">complexity</span><span class="operator">*</span><span class="punctuation">(</span><span class="name">shape</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">]</span><span class="operator">+</span><span class="name">shape</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">]))</span>
    <span class="name">n_density</span> <span class="operator">=</span> <span class="name builtin">int</span><span class="punctuation">(</span><span class="name">density</span><span class="operator">*</span><span class="punctuation">(</span><span class="name">shape</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">]</span><span class="operator">*</span><span class="name">shape</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">]))</span>

    <span class="comment single"># Build actual maze</span>
    <span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">zeros</span><span class="punctuation">(</span><span class="name">shape</span><span class="punctuation">,</span> <span class="name">dtype</span><span class="operator">=</span><span class="name builtin">bool</span><span class="punctuation">)</span>

    <span class="comment single"># Fill borders</span>
    <span class="name">Z</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">,</span> <span class="punctuation">:]</span> <span class="operator">=</span> <span class="name">Z</span><span class="punctuation">[</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">,</span> <span class="punctuation">:]</span> <span class="operator">=</span> <span class="name">Z</span><span class="punctuation">[:,</span> <span class="literal number integer">0</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="name">Z</span><span class="punctuation">[:,</span> <span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="literal number integer">1</span>

    <span class="comment single"># Islands starting point with a bias in favor of border</span>
    <span class="name">P</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">random</span><span class="operator">.</span><span class="name">normal</span><span class="punctuation">(</span><span class="literal number integer">0</span><span class="punctuation">,</span> <span class="literal number float">0.5</span><span class="punctuation">,</span> <span class="punctuation">(</span><span class="name">n_density</span><span class="punctuation">,</span> <span class="literal number integer">2</span><span class="punctuation">))</span>
    <span class="name">P</span> <span class="operator">=</span> <span class="literal number float">0.5</span> <span class="operator">-</span> <span class="name">np</span><span class="operator">.</span><span class="name">maximum</span><span class="punctuation">(</span><span class="operator">-</span><span class="literal number float">0.5</span><span class="punctuation">,</span> <span class="name">np</span><span class="operator">.</span><span class="name">minimum</span><span class="punctuation">(</span><span class="name">P</span><span class="punctuation">,</span> <span class="operator">+</span><span class="literal number float">0.5</span><span class="punctuation">))</span>
    <span class="name">P</span> <span class="operator">=</span> <span class="punctuation">(</span><span class="name">P</span><span class="operator">*</span><span class="punctuation">[</span><span class="name">shape</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">],</span> <span class="name">shape</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">]])</span><span class="operator">.</span><span class="name">astype</span><span class="punctuation">(</span><span class="name builtin">int</span><span class="punctuation">)</span>
    <span class="name">P</span> <span class="operator">=</span> <span class="literal number integer">2</span><span class="operator">*</span><span class="punctuation">(</span><span class="name">P</span><span class="operator">//</span><span class="literal number integer">2</span><span class="punctuation">)</span>

    <span class="comment single"># Create islands</span>
    <span class="keyword">for</span> <span class="name">i</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="name">n_density</span><span class="punctuation">):</span>
        <span class="comment single"># Test for early stop: if all starting point are busy, this means we</span>
        <span class="comment single"># won't be able to connect any island, so we stop.</span>
        <span class="name">T</span> <span class="operator">=</span> <span class="name">Z</span><span class="punctuation">[</span><span class="literal number integer">2</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">2</span><span class="punctuation">:</span><span class="literal number integer">2</span><span class="punctuation">,</span> <span class="literal number integer">2</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">2</span><span class="punctuation">:</span><span class="literal number integer">2</span><span class="punctuation">]</span>
        <span class="keyword">if</span> <span class="name">T</span><span class="operator">.</span><span class="name">sum</span><span class="punctuation">()</span> <span class="operator">==</span> <span class="name">T</span><span class="operator">.</span><span class="name">size</span><span class="punctuation">:</span> <span class="keyword">break</span>
        <span class="name">x</span><span class="punctuation">,</span> <span class="name">y</span> <span class="operator">=</span> <span class="name">P</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">]</span>
        <span class="name">Z</span><span class="punctuation">[</span><span class="name">y</span><span class="punctuation">,</span> <span class="name">x</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="literal number integer">1</span>
        <span class="keyword">for</span> <span class="name">j</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="name">n_complexity</span><span class="punctuation">):</span>
            <span class="name">neighbours</span> <span class="operator">=</span> <span class="punctuation">[]</span>
            <span class="keyword">if</span> <span class="name">x</span> <span class="operator">&gt;</span> <span class="literal number integer">1</span><span class="punctuation">:</span>          <span class="name">neighbours</span><span class="operator">.</span><span class="name">append</span><span class="punctuation">([(</span><span class="name">y</span><span class="punctuation">,</span> <span class="name">x</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">),</span> <span class="punctuation">(</span><span class="name">y</span><span class="punctuation">,</span> <span class="name">x</span><span class="operator">-</span><span class="literal number integer">2</span><span class="punctuation">)])</span>
            <span class="keyword">if</span> <span class="name">x</span> <span class="operator">&lt;</span> <span class="name">shape</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">]</span><span class="operator">-</span><span class="literal number integer">2</span><span class="punctuation">:</span> <span class="name">neighbours</span><span class="operator">.</span><span class="name">append</span><span class="punctuation">([(</span><span class="name">y</span><span class="punctuation">,</span> <span class="name">x</span><span class="operator">+</span><span class="literal number integer">1</span><span class="punctuation">),</span> <span class="punctuation">(</span><span class="name">y</span><span class="punctuation">,</span> <span class="name">x</span><span class="operator">+</span><span class="literal number integer">2</span><span class="punctuation">)])</span>
            <span class="keyword">if</span> <span class="name">y</span> <span class="operator">&gt;</span> <span class="literal number integer">1</span><span class="punctuation">:</span>          <span class="name">neighbours</span><span class="operator">.</span><span class="name">append</span><span class="punctuation">([(</span><span class="name">y</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">,</span> <span class="name">x</span><span class="punctuation">),</span> <span class="punctuation">(</span><span class="name">y</span><span class="operator">-</span><span class="literal number integer">2</span><span class="punctuation">,</span> <span class="name">x</span><span class="punctuation">)])</span>
            <span class="keyword">if</span> <span class="name">y</span> <span class="operator">&lt;</span> <span class="name">shape</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">]</span><span class="operator">-</span><span class="literal number integer">2</span><span class="punctuation">:</span> <span class="name">neighbours</span><span class="operator">.</span><span class="name">append</span><span class="punctuation">([(</span><span class="name">y</span><span class="operator">+</span><span class="literal number integer">1</span><span class="punctuation">,</span> <span class="name">x</span><span class="punctuation">),</span> <span class="punctuation">(</span><span class="name">y</span><span class="operator">+</span><span class="literal number integer">2</span><span class="punctuation">,</span> <span class="name">x</span><span class="punctuation">)])</span>
            <span class="keyword">if</span> <span class="name builtin">len</span><span class="punctuation">(</span><span class="name">neighbours</span><span class="punctuation">):</span>
                <span class="name">choice</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">random</span><span class="operator">.</span><span class="name">randint</span><span class="punctuation">(</span><span class="name builtin">len</span><span class="punctuation">(</span><span class="name">neighbours</span><span class="punctuation">))</span>
                <span class="name">next_1</span><span class="punctuation">,</span> <span class="name">next_2</span> <span class="operator">=</span> <span class="name">neighbours</span><span class="punctuation">[</span><span class="name">choice</span><span class="punctuation">]</span>
                <span class="keyword">if</span> <span class="name">Z</span><span class="punctuation">[</span><span class="name">next_2</span><span class="punctuation">]</span> <span class="operator">==</span> <span class="literal number integer">0</span><span class="punctuation">:</span>
                    <span class="name">Z</span><span class="punctuation">[</span><span class="name">next_1</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="literal number integer">1</span>
                    <span class="name">Z</span><span class="punctuation">[</span><span class="name">next_2</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="literal number integer">1</span>
                    <span class="name">y</span><span class="punctuation">,</span> <span class="name">x</span> <span class="operator">=</span> <span class="name">next_2</span>
            <span class="keyword">else</span><span class="punctuation">:</span>
                <span class="keyword">break</span>
    <span class="keyword">return</span> <span class="name">Z</span>
</pre>
<p>Here is an animation showing the generation process.</p>
<div class="admonition legend">
<p class="first admonition-title"><strong>Figure 5.2</strong></p>
<p class="last">Progressive maze building with complexity and density control.</p>
</div>
<video width="100%" controls>
<source src="data/maze-build.mp4" type="video/mp4">
Your browser does not support the video tag. </video></div>
<div class="section" id="breadth-first">
<h3><a class="toc-backref" href="#id20">Breadth-first</a></h3>
<p>The breadth-first (as well as depth-first) search algorithm addresses the problem
of finding a path between two nodes by examining all possibilities starting
from the root node and stopping as soon as a solution has been found
(destination node has been reached). This algorithm runs in linear time with
complexity in <span class="formula"><i>O</i>(|<i>V</i>| + |<i>E</i>|)</span> (where <span class="formula"><i>V</i></span> is the number of vertices, and <span class="formula"><i>E</i></span> is
the number of edges). Writing such an algorithm is not especially difficult,
provided you have the right data structure. In our case, the array
representation of the maze is not the most well-suited and we need to transform
it into an actual graph as proposed by <a class="reference external" href="http://bryukh.com">Valentin Bryukhanov</a>.</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function">build_graph</span><span class="punctuation">(</span><span class="name">maze</span><span class="punctuation">):</span>
    <span class="name">height</span><span class="punctuation">,</span> <span class="name">width</span> <span class="operator">=</span> <span class="name">maze</span><span class="operator">.</span><span class="name">shape</span>
    <span class="name">graph</span> <span class="operator">=</span> <span class="punctuation">{(</span><span class="name">i</span><span class="punctuation">,</span> <span class="name">j</span><span class="punctuation">):</span> <span class="punctuation">[]</span> <span class="keyword">for</span> <span class="name">j</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="name">width</span><span class="punctuation">)</span>
                        <span class="keyword">for</span> <span class="name">i</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="name">height</span><span class="punctuation">)</span> <span class="keyword">if</span> <span class="operator word">not</span> <span class="name">maze</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">][</span><span class="name">j</span><span class="punctuation">]}</span>
    <span class="keyword">for</span> <span class="name">row</span><span class="punctuation">,</span> <span class="name">col</span> <span class="operator word">in</span> <span class="name">graph</span><span class="operator">.</span><span class="name">keys</span><span class="punctuation">():</span>
        <span class="keyword">if</span> <span class="name">row</span> <span class="operator">&lt;</span> <span class="name">height</span> <span class="operator">-</span> <span class="literal number integer">1</span> <span class="operator word">and</span> <span class="operator word">not</span> <span class="name">maze</span><span class="punctuation">[</span><span class="name">row</span> <span class="operator">+</span> <span class="literal number integer">1</span><span class="punctuation">][</span><span class="name">col</span><span class="punctuation">]:</span>
            <span class="name">graph</span><span class="punctuation">[(</span><span class="name">row</span><span class="punctuation">,</span> <span class="name">col</span><span class="punctuation">)]</span><span class="operator">.</span><span class="name">append</span><span class="punctuation">((</span><span class="literal string double">&quot;S&quot;</span><span class="punctuation">,</span> <span class="punctuation">(</span><span class="name">row</span> <span class="operator">+</span> <span class="literal number integer">1</span><span class="punctuation">,</span> <span class="name">col</span><span class="punctuation">)))</span>
            <span class="name">graph</span><span class="punctuation">[(</span><span class="name">row</span> <span class="operator">+</span> <span class="literal number integer">1</span><span class="punctuation">,</span> <span class="name">col</span><span class="punctuation">)]</span><span class="operator">.</span><span class="name">append</span><span class="punctuation">((</span><span class="literal string double">&quot;N&quot;</span><span class="punctuation">,</span> <span class="punctuation">(</span><span class="name">row</span><span class="punctuation">,</span> <span class="name">col</span><span class="punctuation">)))</span>
        <span class="keyword">if</span> <span class="name">col</span> <span class="operator">&lt;</span> <span class="name">width</span> <span class="operator">-</span> <span class="literal number integer">1</span> <span class="operator word">and</span> <span class="operator word">not</span> <span class="name">maze</span><span class="punctuation">[</span><span class="name">row</span><span class="punctuation">][</span><span class="name">col</span> <span class="operator">+</span> <span class="literal number integer">1</span><span class="punctuation">]:</span>
            <span class="name">graph</span><span class="punctuation">[(</span><span class="name">row</span><span class="punctuation">,</span> <span class="name">col</span><span class="punctuation">)]</span><span class="operator">.</span><span class="name">append</span><span class="punctuation">((</span><span class="literal string double">&quot;E&quot;</span><span class="punctuation">,</span> <span class="punctuation">(</span><span class="name">row</span><span class="punctuation">,</span> <span class="name">col</span> <span class="operator">+</span> <span class="literal number integer">1</span><span class="punctuation">)))</span>
            <span class="name">graph</span><span class="punctuation">[(</span><span class="name">row</span><span class="punctuation">,</span> <span class="name">col</span> <span class="operator">+</span> <span class="literal number integer">1</span><span class="punctuation">)]</span><span class="operator">.</span><span class="name">append</span><span class="punctuation">((</span><span class="literal string double">&quot;W&quot;</span><span class="punctuation">,</span> <span class="punctuation">(</span><span class="name">row</span><span class="punctuation">,</span> <span class="name">col</span><span class="punctuation">)))</span>
    <span class="keyword">return</span> <span class="name">graph</span>
</pre>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">If we had used the depth-first algorithm, there is no guarantee to find the
shortest path, only to find a path (if it exists).</p>
</div>
<p>Once this is done, writing the breadth-first algorithm is
straightforward. We start from the starting node and we visit nodes at
the current depth only (breadth-first, remember?) and we iterate the
process until reaching the final node, if possible. The question is
then: do we get the shortest path exploring the graph this way? In
this specific case, &quot;yes&quot;, because we don't have an edge-weighted graph,
i.e. all the edges have the same weight (or cost).</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function">breadth_first</span><span class="punctuation">(</span><span class="name">maze</span><span class="punctuation">,</span> <span class="name">start</span><span class="punctuation">,</span> <span class="name">goal</span><span class="punctuation">):</span>
    <span class="name">queue</span> <span class="operator">=</span> <span class="name">deque</span><span class="punctuation">([([</span><span class="name">start</span><span class="punctuation">],</span> <span class="name">start</span><span class="punctuation">)])</span>
    <span class="name">visited</span> <span class="operator">=</span> <span class="name builtin">set</span><span class="punctuation">()</span>
    <span class="name">graph</span> <span class="operator">=</span> <span class="name">build_graph</span><span class="punctuation">(</span><span class="name">maze</span><span class="punctuation">)</span>

    <span class="keyword">while</span> <span class="name">queue</span><span class="punctuation">:</span>
        <span class="name">path</span><span class="punctuation">,</span> <span class="name">current</span> <span class="operator">=</span> <span class="name">queue</span><span class="operator">.</span><span class="name">popleft</span><span class="punctuation">()</span>
        <span class="keyword">if</span> <span class="name">current</span> <span class="operator">==</span> <span class="name">goal</span><span class="punctuation">:</span>
            <span class="keyword">return</span> <span class="name">np</span><span class="operator">.</span><span class="name">array</span><span class="punctuation">(</span><span class="name">path</span><span class="punctuation">)</span>
        <span class="keyword">if</span> <span class="name">current</span> <span class="operator word">in</span> <span class="name">visited</span><span class="punctuation">:</span>
            <span class="keyword">continue</span>
        <span class="name">visited</span><span class="operator">.</span><span class="name">add</span><span class="punctuation">(</span><span class="name">current</span><span class="punctuation">)</span>
        <span class="keyword">for</span> <span class="name">direction</span><span class="punctuation">,</span> <span class="name">neighbour</span> <span class="operator word">in</span> <span class="name">graph</span><span class="punctuation">[</span><span class="name">current</span><span class="punctuation">]:</span>
            <span class="name">p</span> <span class="operator">=</span> <span class="name builtin">list</span><span class="punctuation">(</span><span class="name">path</span><span class="punctuation">)</span>
            <span class="name">p</span><span class="operator">.</span><span class="name">append</span><span class="punctuation">(</span><span class="name">neighbour</span><span class="punctuation">)</span>
            <span class="name">queue</span><span class="operator">.</span><span class="name">append</span><span class="punctuation">((</span><span class="name">p</span><span class="punctuation">,</span> <span class="name">neighbour</span><span class="punctuation">))</span>
    <span class="keyword">return</span> <span class="name builtin pseudo">None</span>
</pre>
</div>
<div class="section" id="bellman-ford-method">
<h3><a class="toc-backref" href="#id20">Bellman-Ford method</a></h3>
<p>The Bellman–Ford algorithm is an algorithm that is able to find the optimal
path in a graph using a diffusion process. The optimal path is found by ascending
the resulting gradient. This algorithm runs in quadratic time <span class="formula"><i>O</i>(|<i>V</i>||<i>E</i>|)</span>
(where <span class="formula"><i>V</i></span> is the number of vertices, and <span class="formula"><i>E</i></span> is the number of edges). However, in
our simple case, we won't hit the worst case scenario. The algorithm is
illustrated below (reading from left to right, top to bottom). Once this is
done, we can ascend the gradient from the starting node. You can check on the
figure that this leads to the shortest path.</p>
<div class="admonition legend">
<p class="first admonition-title"><strong>Figure 5.3</strong></p>
<p class="last">Value iteration algorithm on a simple maze. Once entrance has been reached,
it is easy to find the shortest path by ascending the value gradient.</p>
</div>
<img alt="data/value-iteration-1.png" src="data/value-iteration-1.png" style="width: 19%;" />
<img alt="data/value-iteration-2.png" src="data/value-iteration-2.png" style="width: 19%;" />
<img alt="data/value-iteration-3.png" src="data/value-iteration-3.png" style="width: 19%;" />
<img alt="data/value-iteration-4.png" src="data/value-iteration-4.png" style="width: 19%;" />
<img alt="data/value-iteration-5.png" src="data/value-iteration-5.png" style="width: 19%;" />
<img alt="data/value-iteration-6.png" src="data/value-iteration-6.png" style="width: 19%;" />
<img alt="data/value-iteration-7.png" src="data/value-iteration-7.png" style="width: 19%;" />
<img alt="data/value-iteration-8.png" src="data/value-iteration-8.png" style="width: 19%;" />
<img alt="data/value-iteration-9.png" src="data/value-iteration-9.png" style="width: 19%;" />
<img alt="data/value-iteration-10.png" src="data/value-iteration-10.png" style="width: 19%;" />
<p>We start by setting the exit node to the value 1, while every other node is
set to 0, except the walls. Then we iterate a process such that each
cell's new value is computed as the maximum value between the current cell value
and the discounted (<code>gamma=0.9</code> in the case below) 4 neighbour values. The
process starts as soon as the starting node value becomes strictly positive.</p>
<p>The NumPy implementation is straightforward if we take advantage of the
<code>generic_filter</code> (from <code>scipy.ndimage</code>) for the diffusion process:</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function">diffuse</span><span class="punctuation">(</span><span class="name">Z</span><span class="punctuation">):</span>
    <span class="comment single"># North, West, Center, East, South</span>
    <span class="keyword">return</span> <span class="name builtin">max</span><span class="punctuation">(</span><span class="name">gamma</span><span class="operator">*</span><span class="name">Z</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">],</span> <span class="name">gamma</span><span class="operator">*</span><span class="name">Z</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">],</span> <span class="name">Z</span><span class="punctuation">[</span><span class="literal number integer">2</span><span class="punctuation">],</span> <span class="name">gamma</span><span class="operator">*</span><span class="name">Z</span><span class="punctuation">[</span><span class="literal number integer">3</span><span class="punctuation">],</span> <span class="name">gamma</span><span class="operator">*</span><span class="name">Z</span><span class="punctuation">[</span><span class="literal number integer">4</span><span class="punctuation">])</span>

<span class="comment single"># Build gradient array</span>
<span class="name">G</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">zeros</span><span class="punctuation">(</span><span class="name">Z</span><span class="operator">.</span><span class="name">shape</span><span class="punctuation">)</span>

<span class="comment single"># Initialize gradient at the entrance with value 1</span>
<span class="name">G</span><span class="punctuation">[</span><span class="name">start</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="literal number integer">1</span>

<span class="comment single"># Discount factor</span>
<span class="name">gamma</span> <span class="operator">=</span> <span class="literal number float">0.99</span>

<span class="comment single"># We iterate until value at exit is &gt; 0. This requires the maze</span>
<span class="comment single"># to have a solution or it will be stuck in the loop.</span>
<span class="keyword">while</span> <span class="name">G</span><span class="punctuation">[</span><span class="name">goal</span><span class="punctuation">]</span> <span class="operator">==</span> <span class="literal number float">0.0</span><span class="punctuation">:</span>
    <span class="name">G</span> <span class="operator">=</span> <span class="name">Z</span> <span class="operator">*</span> <span class="name">generic_filter</span><span class="punctuation">(</span><span class="name">G</span><span class="punctuation">,</span> <span class="name">diffuse</span><span class="punctuation">,</span> <span class="name">footprint</span><span class="operator">=</span><span class="punctuation">[[</span><span class="literal number integer">0</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">,</span> <span class="literal number integer">0</span><span class="punctuation">],</span>
                                                  <span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">],</span>
                                                  <span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">,</span> <span class="literal number integer">0</span><span class="punctuation">]])</span>
</pre>
<p>But in this specific case, it is rather slow. We'd better cook-up our own
solution, reusing part of the game of life code:</p>
<pre class="code python literal-block">
<span class="comment single"># Build gradient array</span>
<span class="name">G</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">zeros</span><span class="punctuation">(</span><span class="name">Z</span><span class="operator">.</span><span class="name">shape</span><span class="punctuation">)</span>

<span class="comment single"># Initialize gradient at the entrance with value 1</span>
<span class="name">G</span><span class="punctuation">[</span><span class="name">start</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="literal number integer">1</span>

<span class="comment single"># Discount factor</span>
<span class="name">gamma</span> <span class="operator">=</span> <span class="literal number float">0.99</span>

<span class="comment single"># We iterate until value at exit is &gt; 0. This requires the maze</span>
<span class="comment single"># to have a solution or it will be stuck in the loop.</span>
<span class="name">G_gamma</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">empty_like</span><span class="punctuation">(</span><span class="name">G</span><span class="punctuation">)</span>
<span class="keyword">while</span> <span class="name">G</span><span class="punctuation">[</span><span class="name">goal</span><span class="punctuation">]</span> <span class="operator">==</span> <span class="literal number float">0.0</span><span class="punctuation">:</span>
    <span class="name">np</span><span class="operator">.</span><span class="name">multiply</span><span class="punctuation">(</span><span class="name">G</span><span class="punctuation">,</span> <span class="name">gamma</span><span class="punctuation">,</span> <span class="name">out</span><span class="operator">=</span><span class="name">G_gamma</span><span class="punctuation">)</span>
    <span class="name">N</span> <span class="operator">=</span> <span class="name">G_gamma</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">2</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">]</span>
    <span class="name">W</span> <span class="operator">=</span> <span class="name">G_gamma</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">2</span><span class="punctuation">]</span>
    <span class="name">C</span> <span class="operator">=</span> <span class="name">G</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">]</span>
    <span class="name">E</span> <span class="operator">=</span> <span class="name">G_gamma</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">2</span><span class="punctuation">:]</span>
    <span class="name">S</span> <span class="operator">=</span> <span class="name">G_gamma</span><span class="punctuation">[</span><span class="literal number integer">2</span><span class="punctuation">:,</span><span class="literal number integer">1</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">]</span>
    <span class="name">G</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="name">Z</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">:</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">]</span><span class="operator">*</span><span class="name">np</span><span class="operator">.</span><span class="name">maximum</span><span class="punctuation">(</span><span class="name">N</span><span class="punctuation">,</span><span class="name">np</span><span class="operator">.</span><span class="name">maximum</span><span class="punctuation">(</span><span class="name">W</span><span class="punctuation">,</span>
                                <span class="name">np</span><span class="operator">.</span><span class="name">maximum</span><span class="punctuation">(</span><span class="name">C</span><span class="punctuation">,</span><span class="name">np</span><span class="operator">.</span><span class="name">maximum</span><span class="punctuation">(</span><span class="name">E</span><span class="punctuation">,</span><span class="name">S</span><span class="punctuation">))))</span>
</pre>
<p>Once this is done, we can ascend the gradient to find the shortest path as
illustrated on the figure below:</p>
<div class="admonition legend">
<p class="first admonition-title"><strong>Figure 5.4</strong></p>
<p class="last">Path finding using the Bellman-Ford algorithm. Gradient colors indicate
propagated values from the end-point of the maze (bottom-right). Path is
found by ascending gradient from the goal.</p>
</div>
<img alt="data/maze.png" src="data/maze.png" style="width: 100%;" />
</div>
<div class="section" id="id22">
<h3><a class="toc-backref" href="#id20">Sources</a></h3>
<ul class="simple">
<li><a class="reference external" href="code/maze_build.py">maze_build.py</a></li>
<li><a class="reference external" href="code/maze_numpy.py">maze_numpy.py</a></li>
</ul>
</div>
<div class="section" id="id23">
<h3><a class="toc-backref" href="#id20">References</a></h3>
<ul class="simple">
<li><a class="reference external" href="http://bryukh.com/labyrinth-algorithms/">Labyrinth Algorithms</a>, Valentin
Bryukhanov, 2014.</li>
</ul>
</div>
</div>
<div class="section" id="fluid-dynamics">
<h2><a class="toc-backref" href="#id20">Fluid Dynamics</a></h2>
<div class="admonition legend">
<p class="first admonition-title"><strong>Figure 5.5</strong></p>
<p class="last">Hydrodynamic flow at two different zoom levels, Neckar river, Heidelberg,
Germany. Image by <a class="reference external" href="https://commons.wikimedia.org/wiki/File:Self_Similar_Turbulence.png">Steven Mathey</a>, 2012.</p>
</div>
<img alt="data/Self-similar-turbulence.png" src="data/Self-similar-turbulence.png" style="width: 100%;" />
<div class="section" id="lagrangian-vs-eulerian-method">
<h3><a class="toc-backref" href="#id20">Lagrangian vs Eulerian method</a></h3>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">Excerpt from the Wikipedia entry on the
<a class="reference external" href="https://en.wikipedia.org/wiki/Lagrangian_and_Eulerian_specification_of_the_flow_field">Lagrangian and Eulerian specification</a></p>
</div>
<p>In classical field theory, the Lagrangian specification of the field is a way of
looking at fluid motion where the observer follows an individual fluid parcel
as it moves through space and time. Plotting the position of an individual
parcel through time gives the pathline of the parcel. This can be visualized as
sitting in a boat and drifting down a river.</p>
<p>The Eulerian specification of the flow field is a way of looking at fluid
motion that focuses on specific locations in the space through which the fluid
flows as time passes. This can be visualized by sitting on the bank of a river
and watching the water pass the fixed location.</p>
<p>In other words, in the Eulerian case, you divide a portion of space into cells
and each cell contains a velocity vector and other information, such as density
and temperature. In the Lagrangian case, we need particle-based physics with
dynamic interactions and generally we need a high number of particles. Both
methods have advantages and disadvantages and the choice between the two
methods depends on the nature of your problem. Of course, you can also mix the
two methods into a hybrid method.</p>
<p>However, the biggest problem for particle-based simulation is that particle
interaction requires finding neighbouring particles and this has a cost as
we've seen in the boids case. If we target Python and NumPy only, it is probably
better to choose the Eulerian method since vectorization will be almost trivial
compared to the Lagrangian method.</p>
</div>
<div class="section" id="id24">
<h3><a class="toc-backref" href="#id20">NumPy implementation</a></h3>
<p>I won't explain all the theory behind computational fluid dynamics because
first, I cannot (I'm not an expert at all in this domain) and there are many
resources online that explain this nicely (have a look at references below,
especially tutorial by L. Barba). Why choose a computational fluid as an example
then? Because results are (almost) always beautiful and fascinating. I couldn't
resist (look at the movie below).</p>
<p>We'll further simplify the problem by implementing a method from computer
graphics where the goal is not correctness but convincing behavior. Jos Stam
wrote a very nice article for SIGGRAPH 1999 describing a technique to have
stable fluids over time (i.e. whose solution in the long term does not
diverge). <a class="reference external" href="https://github.com/albertosantini/python-fluid">Alberto Santini</a>
wrote a Python replication a long time ago (using numarray!) such that I only
had to adapt it to modern NumPy and accelerate it a bit using modern NumPy
tricks.</p>
<p>I won't comment the code since it would be too long, but you can read the
original paper as well as the explanation by <a class="reference external" href="http://prideout.net/blog/?p=58">Philip Rideout</a> on his blog. Below are some movies I've made
using this technique.</p>
<div class="admonition legend">
<p class="first admonition-title"><strong>Figure 5.6</strong></p>
<p class="last">Smoke simulation using the stable fluids algorithm by Jos Stam.  Right most
video comes from the <a class="reference external" href="http://glumpy.github.io">glumpy</a> package and is
using the GPU (framebuffer operations, i.e. no OpenCL nor CUDA) for faster
computations.</p>
</div>
<video width="33%" controls>
<source src="data/smoke-1.mp4" type="video/mp4">
Your browser does not support the video tag. </video>

<video width="33%" controls>
<source src="data/smoke-2.mp4" type="video/mp4">
Your browser does not support the video tag. </video>

<video width="33%" controls>
<source src="data/smoke-gpu.mp4" type="video/mp4">
Your browser does not support the video tag. </video></div>
<div class="section" id="id25">
<h3><a class="toc-backref" href="#id20">Sources</a></h3>
<ul class="simple">
<li><a class="reference external" href="code/smoke_1.py">smoke_1.py</a></li>
<li><a class="reference external" href="code/smoke_2.py">smoke_2.py</a></li>
<li><a class="reference external" href="code/smoke_solver.py">smoke_solver.py</a></li>
<li><a class="reference external" href="code/smoke_interactive.py">smoke_interactive.py</a></li>
</ul>
</div>
<div class="section" id="id26">
<h3><a class="toc-backref" href="#id20">References</a></h3>
<ul class="simple">
<li><a class="reference external" href="https://github.com/barbagroup/CFDPython">12 Steps to Navier-Stokes</a>, Lorena Barba, 2013.</li>
<li><a class="reference external" href="http://www.dgp.toronto.edu/people/stam/reality/Research/pdf/ns.pdf">Stable Fluids</a>, Jos Stam, 1999.</li>
<li><a class="reference external" href="http://prideout.net/blog/?p=58">Simple Fluid Simulation</a>, Philip Rideout, 2010</li>
<li><a class="reference external" href="http://http.developer.nvidia.com/GPUGems/gpugems_ch38.html">Fast Fluid Dynamics Simulation on the GPU</a>, Mark Harris, 2004.</li>
<li><a class="reference external" href="https://www.cs.ubc.ca/%7Erbridson/docs/zhu-siggraph05-sandfluid.pdf">Animating Sand as a Fluid</a>, Yongning Zhu &amp; Robert Bridson, 2005.</li>
</ul>
</div>
</div>
<div class="section" id="blue-noise-sampling">
<h2><a class="toc-backref" href="#id20">Blue noise sampling</a></h2>
<p>Blue noise refers to sample sets that have random and yet uniform distributions
with absence of any spectral bias. Such noise is very useful in a variety of
graphics applications like rendering, dithering, stippling, etc. Many different
methods have been proposed to achieve such noise, but the most simple is certainly
the DART method.</p>
<div class="admonition legend">
<p class="first admonition-title"><strong>Figure 5.7</strong></p>
<p class="last">Detail of &quot;The Starry Night&quot;, Vincent van Gogh, 1889. The detail has been
resampled using voronoi cells whose centers are a blue noise sample.</p>
</div>
<img alt="data/mosaic.png" class="bordered" src="data/mosaic.png" style="width: 100%;" />
<div class="section" id="dart-method">
<h3><a class="toc-backref" href="#id20">DART method</a></h3>
<p>The DART method is one of the earliest and simplest methods. It works by
sequentially drawing uniform random points and only accepting those that lie at a
minimum distance from every previous accepted sample. This sequential method is
therefore extremely slow because each new candidate needs to be tested against
previous accepted candidates. The more points you accept, the slower the
method is. Let's consider the unit surface and a minimum radius <code>r</code> to be enforced
between each point.</p>
<p>Knowing that the densest packing of circles in the plane is the hexagonal
lattice of the bee's honeycomb, we know this density is <span class="formula"><i>d</i> = <span class="fraction"><span class="ignored">(</span><span class="numerator">1</span><span class="ignored">)/(</span><span class="denominator">6</span><span class="ignored">)</span></span><i>π</i><span class="sqrt"><span class="radical">√</span><span class="ignored">(</span><span class="root">3</span><span class="ignored">)</span></span></span> (in fact <a class="reference external" href="https://en.wikipedia.org/wiki/Circle_packing">I learned it</a> while writing this book).
Considering circles with radius <span class="formula"><i>r</i></span>, we can pack at most <span class="formula"><span class="fraction"><span class="ignored">(</span><span class="numerator"><i>d</i></span><span class="ignored">)/(</span><span class="denominator"><i>π</i><i>r</i><sup>2</sup></span><span class="ignored">)</span></span> = <span class="fraction"><span class="ignored">(</span><span class="numerator"><span class="sqrt"><span class="radical">√</span><span class="ignored">(</span><span class="root">3</span><span class="ignored">)</span></span></span><span class="ignored">)/(</span><span class="denominator">6<i>r</i><sup>2</sup></span><span class="ignored">)</span></span> = <span class="fraction"><span class="ignored">(</span><span class="numerator">1</span><span class="ignored">)/(</span><span class="denominator">2<i>r</i><sup>2</sup><span class="sqrt"><span class="radical">√</span><span class="ignored">(</span><span class="root">3</span><span class="ignored">)</span></span></span><span class="ignored">)</span></span></span>. We know the theoretical
upper limit for the number of discs we can pack onto the surface, but we'll
likely not reach this upper limit because of random placements. Furthermore,
because a lot of points will be rejected after a few have been accepted, we
need to set a limit on the number of successive failed trials before we stop
the whole process.</p>
<pre class="code python literal-block">
<span class="keyword namespace">import</span> <span class="name namespace">math</span>
<span class="keyword namespace">import</span> <span class="name namespace">random</span>

<span class="keyword">def</span> <span class="name function">DART_sampling</span><span class="punctuation">(</span><span class="name">width</span><span class="operator">=</span><span class="literal number float">1.0</span><span class="punctuation">,</span> <span class="name">height</span><span class="operator">=</span><span class="literal number float">1.0</span><span class="punctuation">,</span> <span class="name">r</span> <span class="operator">=</span> <span class="literal number float">0.025</span><span class="punctuation">,</span> <span class="name">k</span><span class="operator">=</span><span class="literal number integer">100</span><span class="punctuation">):</span>
    <span class="keyword">def</span> <span class="name function">distance</span><span class="punctuation">(</span><span class="name">p0</span><span class="punctuation">,</span> <span class="name">p1</span><span class="punctuation">):</span>
        <span class="name">dx</span><span class="punctuation">,</span> <span class="name">dy</span> <span class="operator">=</span> <span class="name">p0</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">]</span><span class="operator">-</span><span class="name">p1</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">],</span> <span class="name">p0</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">]</span><span class="operator">-</span><span class="name">p1</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">]</span>
        <span class="keyword">return</span> <span class="name">math</span><span class="operator">.</span><span class="name">hypot</span><span class="punctuation">(</span><span class="name">dx</span><span class="punctuation">,</span> <span class="name">dy</span><span class="punctuation">)</span>

    <span class="name">points</span> <span class="operator">=</span> <span class="punctuation">[]</span>
    <span class="name">i</span> <span class="operator">=</span> <span class="literal number integer">0</span>
    <span class="name">last_success</span> <span class="operator">=</span> <span class="literal number integer">0</span>
    <span class="keyword">while</span> <span class="name builtin pseudo">True</span><span class="punctuation">:</span>
        <span class="name">x</span> <span class="operator">=</span> <span class="name">random</span><span class="operator">.</span><span class="name">uniform</span><span class="punctuation">(</span><span class="literal number integer">0</span><span class="punctuation">,</span> <span class="name">width</span><span class="punctuation">)</span>
        <span class="name">y</span> <span class="operator">=</span> <span class="name">random</span><span class="operator">.</span><span class="name">uniform</span><span class="punctuation">(</span><span class="literal number integer">0</span><span class="punctuation">,</span> <span class="name">height</span><span class="punctuation">)</span>
        <span class="name">accept</span> <span class="operator">=</span> <span class="name builtin pseudo">True</span>
        <span class="keyword">for</span> <span class="name">p</span> <span class="operator word">in</span> <span class="name">points</span><span class="punctuation">:</span>
            <span class="keyword">if</span> <span class="name">distance</span><span class="punctuation">(</span><span class="name">p</span><span class="punctuation">,</span> <span class="punctuation">(</span><span class="name">x</span><span class="punctuation">,</span> <span class="name">y</span><span class="punctuation">))</span> <span class="operator">&lt;</span> <span class="name">r</span><span class="punctuation">:</span>
                <span class="name">accept</span> <span class="operator">=</span> <span class="name builtin pseudo">False</span>
                <span class="keyword">break</span>
        <span class="keyword">if</span> <span class="name">accept</span> <span class="operator word">is</span> <span class="name builtin pseudo">True</span><span class="punctuation">:</span>
            <span class="name">points</span><span class="operator">.</span><span class="name">append</span><span class="punctuation">((</span><span class="name">x</span><span class="punctuation">,</span> <span class="name">y</span><span class="punctuation">))</span>
            <span class="keyword">if</span> <span class="name">i</span><span class="operator">-</span><span class="name">last_success</span> <span class="operator">&gt;</span> <span class="name">k</span><span class="punctuation">:</span>
                <span class="keyword">break</span>
            <span class="name">last_success</span> <span class="operator">=</span> <span class="name">i</span>
        <span class="name">i</span> <span class="operator">+=</span> <span class="literal number integer">1</span>
    <span class="keyword">return</span> <span class="name">points</span>
</pre>
<p>I left as an exercise the vectorization of the DART method. The idea is to
pre-compute enough uniform random samples as well as paired distances and to
test for their sequential inclusion.</p>
</div>
<div class="section" id="bridson-method">
<h3><a class="toc-backref" href="#id20">Bridson method</a></h3>
<p>If the vectorization of the previous method poses no real difficulty, the speed
improvement is not so good and the quality remains low and dependent on the <code>k</code>
parameter. The higher, the better since it basically governs how hard to try to
insert a new sample. But, when there is already a large number of accepted
samples, only chance allows us to find a position to insert a new sample. We
could increase the <code>k</code> value but this would make the method even slower
without any guarantee in quality. It's time to think out-of-the-box and luckily
enough, Robert Bridson did that for us and proposed a simple yet efficient
method:</p>
<blockquote>
<p><strong>Step 0</strong>. Initialize an n-dimensional background grid for storing samples and
accelerating spatial searches. We pick the cell size to be bounded by
<span class="formula"><span class="fraction"><span class="ignored">(</span><span class="numerator"><i>r</i></span><span class="ignored">)/(</span><span class="denominator"><span class="sqrt"><span class="radical">√</span><span class="ignored">(</span><span class="root"><i>n</i></span><span class="ignored">)</span></span></span><span class="ignored">)</span></span></span>, so that each grid cell will contain at most one
sample, and thus the grid can be implemented as a simple n-dimensional array of
integers: the default −1 indicates no sample, a non-negative integer gives the
index of the sample located in a cell.</p>
<p><strong>Step 1</strong>. Select the initial sample, <span class="formula"><i>x</i><sub>0</sub></span>, randomly chosen uniformly from the
domain. Insert it into the background grid, and initialize the “active list”
(an array of sample indices) with this index (zero).</p>
<p><strong>Step 2</strong>. While the active list is not empty, choose a random index from
it (say <span class="formula"><i>i</i></span>). Generate up to <span class="formula"><i>k</i></span> points chosen uniformly from the
spherical annulus between radius <span class="formula"><i>r</i></span> and <span class="formula">2<i>r</i></span> around
<span class="formula"><i>x</i><sub><i>i</i></sub></span>. For each point in turn, check if it is within distance <span class="formula"><i>r</i></span>
of existing samples (using the background grid to only test nearby
samples). If a point is adequately far from existing samples, emit it as the
next sample and add it to the active list. If after <span class="formula"><i>k</i></span> attempts no
such point is found, instead remove <span class="formula"><i>i</i></span> from the active list.</p>
</blockquote>
<p>Implementation poses no real problem and is left as an exercise for the
reader. Note that not only is this method fast, but it also offers a better
quality (more samples) than the DART method even with a high <span class="formula"><i>k</i></span>
parameter.</p>
<div class="admonition legend">
<p class="first admonition-title"><strong>Figure 5.8</strong></p>
<p class="last">Comparison of uniform, grid-jittered and Bridson sampling.</p>
</div>
<img alt="data/sampling.png" src="data/sampling.png" style="width: 100%;" />
</div>
<div class="section" id="id27">
<h3><a class="toc-backref" href="#id20">Sources</a></h3>
<ul class="simple">
<li><a class="reference external" href="code/DART_sampling_python.py">DART_sampling_python.py</a></li>
<li><a class="reference external" href="code/DART_sampling_numpy.py">DART_sampling_numpy.py</a> (solution to the exercise)</li>
<li><a class="reference external" href="code/Bridson_sampling.py">Bridson_sampling.py</a> (solution to the exercise)</li>
<li><a class="reference external" href="code/sampling.py">sampling.py</a></li>
<li><a class="reference external" href="code/mosaic.py">mosaic.py</a></li>
<li><a class="reference external" href="code/voronoi.py">voronoi.py</a></li>
</ul>
</div>
<div class="section" id="id28">
<h3><a class="toc-backref" href="#id20">References</a></h3>
<ul class="simple">
<li><a class="reference external" href="https://bost.ocks.org/mike/algorithms/">Visualizing Algorithms</a>
Mike Bostock, 2014.</li>
<li><a class="reference external" href="http://www.joesfer.com/?p=108">Stippling and Blue Noise</a>
Jose Esteve, 2012.</li>
<li><a class="reference external" href="http://devmag.org.za/2009/05/03/poisson-disk-sampling/">Poisson Disk Sampling</a>
Herman Tulleken, 2009.</li>
<li><a class="reference external" href="http://www.cs.ubc.ca/~rbridson/docs/bridson-siggraph07-poissondisk.pdf">Fast Poisson Disk Sampling in Arbitrary Dimensions</a>,
Robert Bridson, SIGGRAPH, 2007.</li>
</ul>
</div>
</div>
<div class="section" id="id29">
<h2><a class="toc-backref" href="#id20">Conclusion</a></h2>
<p>The last example we've been studying is indeed a nice example where it is more
important to vectorize the problem rather than to vectorize the code (and too
early). In this specific case we were lucky enough to have the work done for us
but it won't be always the case and in such a case, the temptation might be
high to vectorize the first solution we've found. I hope you're now convinced
it might be a good idea in general to look for alternative solutions once
you've found one. You'll (almost) always improve speed by vectorizing your
code, but in the process, you may miss huge improvements.</p>
</div>
</div>
<div class="section" id="custom-vectorization">
<h1><a class="toc-backref" href="#table-of-contents">Custom vectorization</a></h1>
<div class="contents local topic" id="id30">
<p class="topic-title first"><strong>Contents</strong></p>
<ul class="simple">
<li><a class="reference internal" href="#id31" id="id164">Introduction</a></li>
<li><a class="reference internal" href="#typed-list" id="id165">Typed list</a><ul>
<li><a class="reference internal" href="#creation" id="id166">Creation</a></li>
<li><a class="reference internal" href="#access" id="id167">Access</a></li>
<li><a class="reference internal" href="#id33" id="id168">Exercise</a></li>
<li><a class="reference internal" href="#id34" id="id169">Sources</a></li>
</ul>
</li>
<li><a class="reference internal" href="#memory-aware-array" id="id170">Memory aware array</a><ul>
<li><a class="reference internal" href="#id35" id="id171">Glumpy</a></li>
<li><a class="reference internal" href="#array-subclass" id="id172">Array subclass</a></li>
<li><a class="reference internal" href="#computing-extents" id="id173">Computing extents</a></li>
<li><a class="reference internal" href="#keeping-track-of-pending-data" id="id174">Keeping track of pending data</a></li>
<li><a class="reference internal" href="#id37" id="id175">Sources</a></li>
</ul>
</li>
<li><a class="reference internal" href="#id38" id="id176">Conclusion</a></li>
</ul>
</div>
<div class="section" id="id31">
<h2><a class="toc-backref" href="#id30">Introduction</a></h2>
<p>One of the strengths of NumPy is that it can be used to build new objects or to
<a class="reference external" href="https://docs.scipy.org/doc/numpy/user/basics.subclassing.html">subclass the ndarray</a> object. This
later process is a bit tedious but it is worth the effort because it allows you
to improve the <code>ndarray</code> object to suit your problem. We'll examine in
the following section two real-world cases (typed list and memory-aware array)
that are extensively used in the <a class="reference external" href="http://glumpy.github.io">glumpy</a> project
(that I maintain) while the last one (double precision array) is a more
academic case.</p>
</div>
<div class="section" id="typed-list">
<h2><a class="toc-backref" href="#id30">Typed list</a></h2>
<p>Typed list (also known as ragged array) is a list of items that all have the
same data type (in the sense of NumPy). They offer both the list and the
ndarray API (with some restriction of course) but because their respective APIs may not be
compatible in some cases, we have to make choices. For example, concerning
the <code>+</code> operator, we'll choose to use the NumPy API where the value is added to
each individual item instead of expanding the list by appending a new item
(<code>1</code>).</p>
<pre class="code python literal-block">
<span class="operator">&gt;&gt;&gt;</span> <span class="name">l</span> <span class="operator">=</span> <span class="name">TypedList</span><span class="punctuation">([[</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">2</span><span class="punctuation">],</span> <span class="punctuation">[</span><span class="literal number integer">3</span><span class="punctuation">]])</span>
<span class="operator">&gt;&gt;&gt;</span> <span class="keyword">print</span><span class="punctuation">(</span><span class="name">l</span><span class="punctuation">)</span>
<span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">,</span> <span class="literal number integer">2</span><span class="punctuation">],</span> <span class="punctuation">[</span><span class="literal number integer">3</span><span class="punctuation">]</span>
<span class="operator">&gt;&gt;&gt;</span> <span class="keyword">print</span><span class="punctuation">(</span><span class="name">l</span><span class="operator">+</span><span class="literal number integer">1</span><span class="punctuation">)</span>
<span class="punctuation">[</span><span class="literal number integer">2</span><span class="punctuation">,</span> <span class="literal number integer">3</span><span class="punctuation">],</span> <span class="punctuation">[</span><span class="literal number integer">4</span><span class="punctuation">]</span>
</pre>
<p>From the list API, we want our new object to offer the possibility of inserting,
appending and removing items seamlessly.</p>
<div class="section" id="creation">
<h3><a class="toc-backref" href="#id30">Creation</a></h3>
<p>Since the object is dynamic by definition, it is important to offer a
general-purpose creation method powerful enough to avoid having to do later
manipulations. Such manipulations, for example insertion/deletion, cost
a lot of operations and we want to avoid them. Here is a proposal (among
others) for the creation of a <code>TypedList</code> object.</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function magic">__init__</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="punctuation">,</span> <span class="name">data</span><span class="operator">=</span><span class="name builtin pseudo">None</span><span class="punctuation">,</span> <span class="name">sizes</span><span class="operator">=</span><span class="name builtin pseudo">None</span><span class="punctuation">,</span> <span class="name">dtype</span><span class="operator">=</span><span class="name builtin">float</span><span class="punctuation">)</span>
    <span class="literal string doc">&quot;&quot;&quot;
    Parameters
    ----------

    data : array_like
        An array, any object exposing the array interface, an object
        whose __array__ method returns an array, or any (nested) sequence.

    sizes:  int or 1-D array
        If `itemsize is an integer, N, the array will be divided
        into elements of size N. If such partition is not possible,
        an error is raised.

        If `itemsize` is 1-D array, the array will be divided into
        elements whose successive sizes will be picked from itemsize.
        If the sum of itemsize values is different from array size,
        an error is raised.

    dtype: np.dtype
        Any object that can be interpreted as a NumPy data type.
    &quot;&quot;&quot;</span>
</pre>
<p>This API allows creating an empty list or creating a list from some external
data. Note that in the latter case, we need to specify how to partition the
data into several items or they will split into 1-size items. It can be a regular
partition (i.e. each item is 2 data long) or a custom one (i.e. data must be
split in items of size 1, 2, 3 and 4 items).</p>
<pre class="code python literal-block">
<span class="operator">&gt;&gt;&gt;</span> <span class="name">L</span> <span class="operator">=</span> <span class="name">TypedList</span><span class="punctuation">([[</span><span class="literal number integer">0</span><span class="punctuation">],</span> <span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">2</span><span class="punctuation">],</span> <span class="punctuation">[</span><span class="literal number integer">3</span><span class="punctuation">,</span><span class="literal number integer">4</span><span class="punctuation">,</span><span class="literal number integer">5</span><span class="punctuation">],</span> <span class="punctuation">[</span><span class="literal number integer">6</span><span class="punctuation">,</span><span class="literal number integer">7</span><span class="punctuation">,</span><span class="literal number integer">8</span><span class="punctuation">,</span><span class="literal number integer">9</span><span class="punctuation">]])</span>
<span class="operator">&gt;&gt;&gt;</span> <span class="keyword">print</span><span class="punctuation">(</span><span class="name">L</span><span class="punctuation">)</span>
<span class="punctuation">[</span> <span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">]</span> <span class="punctuation">[</span><span class="literal number integer">1</span> <span class="literal number integer">2</span><span class="punctuation">]</span> <span class="punctuation">[</span><span class="literal number integer">3</span> <span class="literal number integer">4</span> <span class="literal number integer">5</span><span class="punctuation">]</span> <span class="punctuation">[</span><span class="literal number integer">6</span> <span class="literal number integer">7</span> <span class="literal number integer">8</span><span class="punctuation">]</span> <span class="punctuation">]</span>

<span class="operator">&gt;&gt;&gt;</span> <span class="name">L</span> <span class="operator">=</span> <span class="name">TypedList</span><span class="punctuation">(</span><span class="name">np</span><span class="operator">.</span><span class="name">arange</span><span class="punctuation">(</span><span class="literal number integer">10</span><span class="punctuation">),</span> <span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">2</span><span class="punctuation">,</span><span class="literal number integer">3</span><span class="punctuation">,</span><span class="literal number integer">4</span><span class="punctuation">])</span>
<span class="punctuation">[</span> <span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">]</span> <span class="punctuation">[</span><span class="literal number integer">1</span> <span class="literal number integer">2</span><span class="punctuation">]</span> <span class="punctuation">[</span><span class="literal number integer">3</span> <span class="literal number integer">4</span> <span class="literal number integer">5</span><span class="punctuation">]</span> <span class="punctuation">[</span><span class="literal number integer">6</span> <span class="literal number integer">7</span> <span class="literal number integer">8</span><span class="punctuation">]</span> <span class="punctuation">]</span>
</pre>
<p>At this point, the question is whether to subclass the <code>ndarray</code> class or to use
an internal <code>ndarray</code> to store our data. In our specific case, it does not really make
sense to subclass <code>ndarray</code> because we don't really want to offer the
<code>ndarray</code> interface. Instead, we'll use an <code>ndarray</code> for storing the list data and
this design choice will offer us more flexibility.</p>
<pre class="code output literal-block">
╌╌╌╌┬───┐┌───┬───┐┌───┬───┬───┐┌───┬───┬───┬───┬╌╌╌╌╌
    │ 0 ││ 1 │ 2 ││ 3 │ 4 │ 5 ││ 6 │ 7 │ 8 │ 9 │
 ╌╌╌┴───┘└───┴───┘└───┴───┴───┘└───┴───┴───┴───┴╌╌╌╌╌╌
   item 1  item 2    item 3         item 4
</pre>
<p>To store the limit of each item, we'll use an <code>items</code> array that will take care
of storing the position (start and end) for each item. For the creation of a
list, there are two distinct cases: no data is given or some data is given. The
first case is easy and requires only the creation of the <code>_data</code> and <code>_items</code>
arrays. Note that their size is not <code>null</code> since it would be too costly to resize
the array each time we insert a new item. Instead, it's better to reserve some
space.</p>
<p><strong>First case.</strong> No data has been given, only dtype.</p>
<pre class="code python literal-block">
<span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_data</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">zeros</span><span class="punctuation">(</span><span class="literal number integer">512</span><span class="punctuation">,</span> <span class="name">dtype</span><span class="operator">=</span><span class="name">dtype</span><span class="punctuation">)</span>
<span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_items</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">zeros</span><span class="punctuation">((</span><span class="literal number integer">64</span><span class="punctuation">,</span><span class="literal number integer">2</span><span class="punctuation">),</span> <span class="name">dtype</span><span class="operator">=</span><span class="name builtin">int</span><span class="punctuation">)</span>
<span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_size</span> <span class="operator">=</span> <span class="literal number integer">0</span>
<span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_count</span> <span class="operator">=</span> <span class="literal number integer">0</span>
</pre>
<p><strong>Second case.</strong> Some data has been given as well as a list of item sizes (for
other cases, see full code below)</p>
<pre class="code python literal-block">
<span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_data</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">array</span><span class="punctuation">(</span><span class="name">data</span><span class="punctuation">,</span> <span class="name">copy</span><span class="operator">=</span><span class="name builtin pseudo">False</span><span class="punctuation">)</span>
<span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_size</span> <span class="operator">=</span> <span class="name">data</span><span class="operator">.</span><span class="name">size</span>
<span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_count</span> <span class="operator">=</span> <span class="name builtin">len</span><span class="punctuation">(</span><span class="name">sizes</span><span class="punctuation">)</span>
<span class="name">indices</span> <span class="operator">=</span> <span class="name">sizes</span><span class="operator">.</span><span class="name">cumsum</span><span class="punctuation">()</span>
<span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_items</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">zeros</span><span class="punctuation">((</span><span class="name builtin">len</span><span class="punctuation">(</span><span class="name">sizes</span><span class="punctuation">),</span><span class="literal number integer">2</span><span class="punctuation">),</span><span class="name builtin">int</span><span class="punctuation">)</span>
<span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_items</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">:,</span><span class="literal number integer">0</span><span class="punctuation">]</span> <span class="operator">+=</span> <span class="name">indices</span><span class="punctuation">[:</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">]</span>
<span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_items</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">:,</span><span class="literal number integer">1</span><span class="punctuation">]</span> <span class="operator">+=</span> <span class="name">indices</span>
</pre>
</div>
<div class="section" id="access">
<h3><a class="toc-backref" href="#id30">Access</a></h3>
<p>Once this is done, every list method requires only a bit of computation and
playing with the different key when getting, inserting or setting an item. Here is
the code for the <code>__getitem__</code> method. No real difficulty but the possible
negative step:</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function magic">__getitem__</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="punctuation">,</span> <span class="name">key</span><span class="punctuation">):</span>
    <span class="keyword">if</span> <span class="name builtin">type</span><span class="punctuation">(</span><span class="name">key</span><span class="punctuation">)</span> <span class="operator word">is</span> <span class="name builtin">int</span><span class="punctuation">:</span>
        <span class="keyword">if</span> <span class="name">key</span> <span class="operator">&lt;</span> <span class="literal number integer">0</span><span class="punctuation">:</span>
            <span class="name">key</span> <span class="operator">+=</span> <span class="name builtin">len</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="punctuation">)</span>
        <span class="keyword">if</span> <span class="name">key</span> <span class="operator">&lt;</span> <span class="literal number integer">0</span> <span class="operator word">or</span> <span class="name">key</span> <span class="operator">&gt;=</span> <span class="name builtin">len</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="punctuation">):</span>
            <span class="keyword">raise</span> <span class="name exception">IndexError</span><span class="punctuation">(</span><span class="literal string double">&quot;Tuple index out of range&quot;</span><span class="punctuation">)</span>
        <span class="name">dstart</span> <span class="operator">=</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_items</span><span class="punctuation">[</span><span class="name">key</span><span class="punctuation">][</span><span class="literal number integer">0</span><span class="punctuation">]</span>
        <span class="name">dstop</span>  <span class="operator">=</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_items</span><span class="punctuation">[</span><span class="name">key</span><span class="punctuation">][</span><span class="literal number integer">1</span><span class="punctuation">]</span>
        <span class="keyword">return</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_data</span><span class="punctuation">[</span><span class="name">dstart</span><span class="punctuation">:</span><span class="name">dstop</span><span class="punctuation">]</span>

    <span class="keyword">elif</span> <span class="name builtin">type</span><span class="punctuation">(</span><span class="name">key</span><span class="punctuation">)</span> <span class="operator word">is</span> <span class="name builtin">slice</span><span class="punctuation">:</span>
        <span class="name">istart</span><span class="punctuation">,</span> <span class="name">istop</span><span class="punctuation">,</span> <span class="name">step</span> <span class="operator">=</span> <span class="name">key</span><span class="operator">.</span><span class="name">indices</span><span class="punctuation">(</span><span class="name builtin">len</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="punctuation">))</span>
        <span class="keyword">if</span> <span class="name">istart</span> <span class="operator">&gt;</span> <span class="name">istop</span><span class="punctuation">:</span>
            <span class="name">istart</span><span class="punctuation">,</span><span class="name">istop</span> <span class="operator">=</span> <span class="name">istop</span><span class="punctuation">,</span><span class="name">istart</span>
        <span class="name">dstart</span> <span class="operator">=</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_items</span><span class="punctuation">[</span><span class="name">istart</span><span class="punctuation">][</span><span class="literal number integer">0</span><span class="punctuation">]</span>
        <span class="keyword">if</span> <span class="name">istart</span> <span class="operator">==</span> <span class="name">istop</span><span class="punctuation">:</span>
            <span class="name">dstop</span> <span class="operator">=</span> <span class="name">dstart</span>
        <span class="keyword">else</span><span class="punctuation">:</span>
            <span class="name">dstop</span>  <span class="operator">=</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_items</span><span class="punctuation">[</span><span class="name">istop</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">][</span><span class="literal number integer">1</span><span class="punctuation">]</span>
        <span class="keyword">return</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_data</span><span class="punctuation">[</span><span class="name">dstart</span><span class="punctuation">:</span><span class="name">dstop</span><span class="punctuation">]</span>

    <span class="keyword">elif</span> <span class="name builtin">isinstance</span><span class="punctuation">(</span><span class="name">key</span><span class="punctuation">,</span><span class="name builtin">str</span><span class="punctuation">):</span>
        <span class="keyword">return</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_data</span><span class="punctuation">[</span><span class="name">key</span><span class="punctuation">][:</span><span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_size</span><span class="punctuation">]</span>

    <span class="keyword">elif</span> <span class="name">key</span> <span class="operator word">is</span> <span class="name builtin pseudo">Ellipsis</span><span class="punctuation">:</span>
        <span class="keyword">return</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">data</span>

    <span class="keyword">else</span><span class="punctuation">:</span>
        <span class="keyword">raise</span> <span class="name exception">TypeError</span><span class="punctuation">(</span><span class="literal string double">&quot;List indices must be integers&quot;</span><span class="punctuation">)</span>
</pre>
</div>
<div class="section" id="id33">
<h3><a class="toc-backref" href="#id30">Exercise</a></h3>
<p>Modification of the list is a bit more complicated, because it requires
managing memory properly. Since it poses no real difficulty, we left this as an
exercise for the reader. For the lazy, you can have a look at the code below.
Be careful with negative steps, key range and array expansion. When the
underlying array needs to be expanded, it's better to expand it more than
necessary in order to avoid future expansion.</p>
<p><strong>setitem</strong></p>
<pre class="code python literal-block">
<span class="name">L</span> <span class="operator">=</span> <span class="name">TypedList</span><span class="punctuation">([[</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">],</span> <span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">],</span> <span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">]])</span>
<span class="name">L</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">1</span>
</pre>
<pre class="code output literal-block">
╌╌╌╌┬───┬───┐┌───┬───┐┌───┬───┬╌╌╌╌╌
    │ 0 │ 0 ││ 1 │ 1 ││ 2 │ 2 │
 ╌╌╌┴───┴───┘└───┴───┘└───┴───┴╌╌╌╌╌╌
     item 1   item 2   item 3

╌╌╌╌┬───┬───┐┌───┬───┲━━━┓┌───┬───┬╌╌╌╌╌
    │ 0 │ 0 ││ 1 │ 1 ┃ 1 ┃│ 2 │ 2 │
 ╌╌╌┴───┴───┘└───┴───┺━━━┛└───┴───┴╌╌╌╌╌╌
     item 1     item 2     item 3
</pre>
<p><strong>delitem</strong></p>
<pre class="code python literal-block">
<span class="name">L</span> <span class="operator">=</span> <span class="name">TypedList</span><span class="punctuation">([[</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">],</span> <span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">],</span> <span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">]])</span>
<span class="keyword">del</span> <span class="name">L</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">]</span>
</pre>
<pre class="code output literal-block">
╌╌╌╌┬───┬───┐┏━━━┳━━━┓┌───┬───┬╌╌╌╌╌
    │ 0 │ 0 │┃ 1 ┃ 1 ┃│ 2 │ 2 │
 ╌╌╌┴───┴───┘┗━━━┻━━━┛└───┴───┴╌╌╌╌╌╌
     item 1   item 2   item 3

╌╌╌╌┬───┬───┐┌───┬───┬╌╌╌╌╌
    │ 0 │ 0 ││ 2 │ 2 │
 ╌╌╌┴───┴───┘└───┴───┴╌╌╌╌╌╌
     item 1    item 2
</pre>
<p><strong>insert</strong></p>
<pre class="code python literal-block">
<span class="name">L</span> <span class="operator">=</span> <span class="name">TypedList</span><span class="punctuation">([[</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">],</span> <span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">],</span> <span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">]])</span>
<span class="name">L</span><span class="operator">.</span><span class="name">insert</span><span class="punctuation">(</span><span class="literal number integer">1</span><span class="punctuation">,</span> <span class="punctuation">[</span><span class="literal number integer">3</span><span class="punctuation">,</span><span class="literal number integer">3</span><span class="punctuation">])</span>
</pre>
<pre class="code output literal-block">
╌╌╌╌┬───┬───┐┌───┬───┐┌───┬───┬╌╌╌╌╌
    │ 0 │ 0 ││ 1 │ 1 ││ 2 │ 2 │
 ╌╌╌┴───┴───┘└───┴───┘└───┴───┴╌╌╌╌╌╌
     item 1   item 2   item 3

╌╌╌╌┬───┬───┐┏━━━┳━━━┓┌───┬───┐┌───┬───┬╌╌╌╌╌
    │ 0 │ 0 │┃ 3 ┃ 3 ┃│ 1 │ 1 ││ 2 │ 2 │
 ╌╌╌┴───┴───┘┗━━━┻━━━┛└───┴───┘└───┴───┴╌╌╌╌╌╌
     item 1   item 2   item 3   item 4
</pre>
</div>
<div class="section" id="id34">
<h3><a class="toc-backref" href="#id30">Sources</a></h3>
<ul class="simple">
<li><a class="reference external" href="code/array_list.py">array_list.py</a> (solution to the exercise)</li>
</ul>
</div>
</div>
<div class="section" id="memory-aware-array">
<h2><a class="toc-backref" href="#id30">Memory aware array</a></h2>
<div class="section" id="id35">
<h3><a class="toc-backref" href="#id30">Glumpy</a></h3>
<p><a class="reference external" href="http://glumpy.github.io">Glumpy</a> is an OpenGL-based interactive
visualization library in Python whose goal is to make it easy to create fast,
scalable, beautiful, interactive and dynamic visualizations.</p>
<div class="admonition legend">
<p class="first admonition-title"><strong>Figure 6.1</strong></p>
<p class="last">Simulation of a spiral galaxy using the density wave theory.</p>
</div>
<img alt="data/galaxy.png" class="bordered" src="data/galaxy.png" style="width: 100%;" />
<div class="line-block">
<div class="line"><br /></div>
</div>
<div class="admonition legend">
<p class="first admonition-title"><strong>Figure 6.2</strong></p>
<p class="last">Tiger display using collections and 2 GL calls</p>
</div>
<img alt="data/tiger.png" class="bordered" src="data/tiger.png" style="width: 100%;" />
<p>Glumpy is based on a tight and seamless integration with NumPy arrays. This
means you can manipulate GPU data as you would with regular NumPy arrays and
glumpy will take care of the rest. But an example is worth a thousand words:</p>
<pre class="code literal-block">
from glumpy import gloo

dtype = [(&quot;position&quot;, np.float32, 2),  # x,y
         (&quot;color&quot;,    np.float32, 3)]  # r,g,b
V = np.zeros((3,3),dtype).view(gloo.VertexBuffer)
V[&quot;position&quot;][0,0] = 0.0, 0.0
V[&quot;position&quot;][1,1] = 0.0, 0.0
</pre>
<p><code>V</code> is a <code>VertexBuffer</code> which is both a <code>GPUData</code> and a NumPy array. When <code>V</code> is
modified, glumpy takes care of computing the smallest contiguous block of dirty
memory since it was last uploaded to GPU memory. When this buffer is to be used
on the GPU, glumpy takes care of uploading the &quot;dirty&quot; area at the very last
moment. This means that if you never use <code>V</code>, nothing will be ever uploaded to
the GPU! In the case above, the last computed &quot;dirty&quot; area is made of 88 bytes
starting at offset 0 as illustrated below:</p>
<img alt="data/GPUData.png" src="data/GPUData.png" style="width: 100%;" />
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">When a buffer is created, it is marked as totally dirty, but for the sake of
illustration, just pretend this is not the case here.</p>
</div>
<p>Glumpy will thus end up uploading 88 bytes while only 16 bytes have been
actually modified. You might wonder if this optimal. Actually, most of the time
it is, because uploading some data to a buffer requires a lot of operations on
the GL side and each call has a fixed cost.</p>
<!-- In the glumpy package, GPU data is the base class for any data that needs to co-exist on both CPU and GPU memory. It keeps track of the smallest contiguous area that needs to be uploaded to GPU to keep the CPU and GPU data synced. This allows to update the data in one operation. Even though this might be sub-optimal in a few cases, it provides a greater usage flexibility and most of the time, it will be faster. This is done transparently and user can use a GPU buffer as a regular NumPy array. The `pending_data` property indicates the region (offset/nbytes) of the base array that needs to be uploaded. -->
</div>
<div class="section" id="array-subclass">
<h3><a class="toc-backref" href="#id30">Array subclass</a></h3>
<p>As explained in the <a class="reference external" href="https://docs.scipy.org/doc/numpy/user/basics.subclassing.html">Subclassing ndarray</a>
documentation, subclassing <code>ndarray</code> is complicated by the fact that new
instances of <code>ndarray</code> classes can come about in three different ways:</p>
<ul class="simple">
<li>Explicit constructor call</li>
<li>View casting</li>
<li>New from template</li>
</ul>
<p>However our case is simpler because we're only interested in the view
casting. We thus only need to define the <code>__new__</code> method that will be called
at each instance creation. As such, the <code>GPUData</code> class will be equipped with two
properties:</p>
<ul class="simple">
<li><code>extents</code>: This represents the full extent of the view relatively to the base
array. It is stored as a byte offset and a byte size.</li>
<li><code>pending_data</code>: This represents the contiguous <em>dirty</em> area as (byte offset,
byte size) relatively to the <code>extents</code> property.</li>
</ul>
<pre class="code python literal-block">
<span class="keyword">class</span> <span class="name class">GPUData</span><span class="punctuation">(</span><span class="name">np</span><span class="operator">.</span><span class="name">ndarray</span><span class="punctuation">):</span>
    <span class="keyword">def</span> <span class="name function magic">__new__</span><span class="punctuation">(</span><span class="name builtin pseudo">cls</span><span class="punctuation">,</span> <span class="operator">*</span><span class="name">args</span><span class="punctuation">,</span> <span class="operator">**</span><span class="name">kwargs</span><span class="punctuation">):</span>
        <span class="keyword">return</span> <span class="name">np</span><span class="operator">.</span><span class="name">ndarray</span><span class="operator">.</span><span class="name function magic">__new__</span><span class="punctuation">(</span><span class="name builtin pseudo">cls</span><span class="punctuation">,</span> <span class="operator">*</span><span class="name">args</span><span class="punctuation">,</span> <span class="operator">**</span><span class="name">kwargs</span><span class="punctuation">)</span>

    <span class="keyword">def</span> <span class="name function magic">__init__</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="punctuation">,</span> <span class="operator">*</span><span class="name">args</span><span class="punctuation">,</span> <span class="operator">**</span><span class="name">kwargs</span><span class="punctuation">):</span>
        <span class="keyword">pass</span>

    <span class="keyword">def</span> <span class="name function">__array_finalize__</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="punctuation">,</span> <span class="name">obj</span><span class="punctuation">):</span>
        <span class="keyword">if</span> <span class="operator word">not</span> <span class="name builtin">isinstance</span><span class="punctuation">(</span><span class="name">obj</span><span class="punctuation">,</span> <span class="name">GPUData</span><span class="punctuation">):</span>
            <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_extents</span> <span class="operator">=</span> <span class="literal number integer">0</span><span class="punctuation">,</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">size</span><span class="operator">*</span><span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">itemsize</span>
            <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name variable magic">__class__</span><span class="operator">.</span><span class="name function magic">__init__</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="punctuation">)</span>
            <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_pending_data</span> <span class="operator">=</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_extents</span>
        <span class="keyword">else</span><span class="punctuation">:</span>
            <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_extents</span> <span class="operator">=</span> <span class="name">obj</span><span class="operator">.</span><span class="name">_extents</span>
</pre>
</div>
<div class="section" id="computing-extents">
<h3><a class="toc-backref" href="#id30">Computing extents</a></h3>
<p>Each time a partial view of the array is requested, we need to compute the
extents of this partial view while we have access to the base array.</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function magic">__getitem__</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="punctuation">,</span> <span class="name">key</span><span class="punctuation">):</span>
    <span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">ndarray</span><span class="operator">.</span><span class="name function magic">__getitem__</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="punctuation">,</span> <span class="name">key</span><span class="punctuation">)</span>
    <span class="keyword">if</span> <span class="operator word">not</span> <span class="name builtin">hasattr</span><span class="punctuation">(</span><span class="name">Z</span><span class="punctuation">,</span><span class="literal string single">'shape'</span><span class="punctuation">)</span> <span class="operator word">or</span> <span class="name">Z</span><span class="operator">.</span><span class="name">shape</span> <span class="operator">==</span> <span class="punctuation">():</span>
        <span class="keyword">return</span> <span class="name">Z</span>
    <span class="name">Z</span><span class="operator">.</span><span class="name">_extents</span> <span class="operator">=</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_compute_extents</span><span class="punctuation">(</span><span class="name">Z</span><span class="punctuation">)</span>
    <span class="keyword">return</span> <span class="name">Z</span>

<span class="keyword">def</span> <span class="name function">_compute_extents</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="punctuation">,</span> <span class="name">Z</span><span class="punctuation">):</span>
    <span class="keyword">if</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">base</span> <span class="operator word">is</span> <span class="operator word">not</span> <span class="name builtin pseudo">None</span><span class="punctuation">:</span>
        <span class="name">base</span> <span class="operator">=</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">base</span><span class="operator">.</span><span class="name">__array_interface__</span><span class="punctuation">[</span><span class="literal string single">'data'</span><span class="punctuation">][</span><span class="literal number integer">0</span><span class="punctuation">]</span>
        <span class="name">view</span> <span class="operator">=</span> <span class="name">Z</span><span class="operator">.</span><span class="name">__array_interface__</span><span class="punctuation">[</span><span class="literal string single">'data'</span><span class="punctuation">][</span><span class="literal number integer">0</span><span class="punctuation">]</span>
        <span class="name">offset</span> <span class="operator">=</span> <span class="name">view</span> <span class="operator">-</span> <span class="name">base</span>
        <span class="name">shape</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">array</span><span class="punctuation">(</span><span class="name">Z</span><span class="operator">.</span><span class="name">shape</span><span class="punctuation">)</span> <span class="operator">-</span> <span class="literal number integer">1</span>
        <span class="name">strides</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">array</span><span class="punctuation">(</span><span class="name">Z</span><span class="operator">.</span><span class="name">strides</span><span class="punctuation">)</span>
        <span class="name">size</span> <span class="operator">=</span> <span class="punctuation">(</span><span class="name">shape</span><span class="operator">*</span><span class="name">strides</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">sum</span><span class="punctuation">()</span> <span class="operator">+</span> <span class="name">Z</span><span class="operator">.</span><span class="name">itemsize</span>
        <span class="keyword">return</span> <span class="name">offset</span><span class="punctuation">,</span> <span class="name">offset</span><span class="operator">+</span><span class="name">size</span>
    <span class="keyword">else</span><span class="punctuation">:</span>
        <span class="keyword">return</span> <span class="literal number integer">0</span><span class="punctuation">,</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">size</span><span class="operator">*</span><span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">itemsize</span>
</pre>
</div>
<div class="section" id="keeping-track-of-pending-data">
<h3><a class="toc-backref" href="#id30">Keeping track of pending data</a></h3>
<p>One extra difficulty is that we don't want all the views to keep track of the
dirty area but only the base array. This is the reason why we don't instantiate
the <code>self._pending_data</code> in the second case of the <code>__array_finalize__</code>
method. This will be handled when we need to update some data as during a
<code>__setitem__</code> call for example:</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function magic">__setitem__</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="punctuation">,</span> <span class="name">key</span><span class="punctuation">,</span> <span class="name">value</span><span class="punctuation">):</span>
    <span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">ndarray</span><span class="operator">.</span><span class="name function magic">__getitem__</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="punctuation">,</span> <span class="name">key</span><span class="punctuation">)</span>
    <span class="keyword">if</span> <span class="name">Z</span><span class="operator">.</span><span class="name">shape</span> <span class="operator">==</span> <span class="punctuation">():</span>
        <span class="name">key</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">mod</span><span class="punctuation">(</span><span class="name">np</span><span class="operator">.</span><span class="name">array</span><span class="punctuation">(</span><span class="name">key</span><span class="punctuation">)</span><span class="operator">+</span><span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">shape</span><span class="punctuation">,</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">shape</span><span class="punctuation">)</span>
        <span class="name">offset</span> <span class="operator">=</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_extents</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">]</span><span class="operator">+</span><span class="punctuation">(</span><span class="name">key</span> <span class="operator">*</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">strides</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">sum</span><span class="punctuation">()</span>
        <span class="name">size</span> <span class="operator">=</span> <span class="name">Z</span><span class="operator">.</span><span class="name">itemsize</span>
        <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_add_pending_data</span><span class="punctuation">(</span><span class="name">offset</span><span class="punctuation">,</span> <span class="name">offset</span><span class="operator">+</span><span class="name">size</span><span class="punctuation">)</span>
        <span class="name">key</span> <span class="operator">=</span> <span class="name builtin">tuple</span><span class="punctuation">(</span><span class="name">key</span><span class="punctuation">)</span>
    <span class="keyword">else</span><span class="punctuation">:</span>
        <span class="name">Z</span><span class="operator">.</span><span class="name">_extents</span> <span class="operator">=</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_compute_extents</span><span class="punctuation">(</span><span class="name">Z</span><span class="punctuation">)</span>
        <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_add_pending_data</span><span class="punctuation">(</span><span class="name">Z</span><span class="operator">.</span><span class="name">_extents</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">],</span> <span class="name">Z</span><span class="operator">.</span><span class="name">_extents</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">])</span>
    <span class="name">np</span><span class="operator">.</span><span class="name">ndarray</span><span class="operator">.</span><span class="name function magic">__setitem__</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="punctuation">,</span> <span class="name">key</span><span class="punctuation">,</span> <span class="name">value</span><span class="punctuation">)</span>

<span class="keyword">def</span> <span class="name function">_add_pending_data</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="punctuation">,</span> <span class="name">start</span><span class="punctuation">,</span> <span class="name">stop</span><span class="punctuation">):</span>
    <span class="name">base</span> <span class="operator">=</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">base</span>
    <span class="keyword">if</span> <span class="name builtin">isinstance</span><span class="punctuation">(</span><span class="name">base</span><span class="punctuation">,</span> <span class="name">GPUData</span><span class="punctuation">):</span>
        <span class="name">base</span><span class="operator">.</span><span class="name">_add_pending_data</span><span class="punctuation">(</span><span class="name">start</span><span class="punctuation">,</span> <span class="name">stop</span><span class="punctuation">)</span>
    <span class="keyword">else</span><span class="punctuation">:</span>
        <span class="keyword">if</span> <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_pending_data</span> <span class="operator word">is</span> <span class="name builtin pseudo">None</span><span class="punctuation">:</span>
            <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_pending_data</span> <span class="operator">=</span> <span class="name">start</span><span class="punctuation">,</span> <span class="name">stop</span>
        <span class="keyword">else</span><span class="punctuation">:</span>
            <span class="name">start</span> <span class="operator">=</span> <span class="name builtin">min</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_pending_data</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">],</span> <span class="name">start</span><span class="punctuation">)</span>
            <span class="name">stop</span> <span class="operator">=</span> <span class="name builtin">max</span><span class="punctuation">(</span><span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_pending_data</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">],</span> <span class="name">stop</span><span class="punctuation">)</span>
            <span class="name builtin pseudo">self</span><span class="operator">.</span><span class="name">_pending_data</span> <span class="operator">=</span> <span class="name">start</span><span class="punctuation">,</span> <span class="name">stop</span>
</pre>
</div>
<div class="section" id="id37">
<h3><a class="toc-backref" href="#id30">Sources</a></h3>
<ul class="simple">
<li><a class="reference external" href="code/gpudata.py">gpudata.py</a></li>
</ul>
<!-- Double precision array -->
<!-- - - - - - - - - - - - - - - - - - - - - - - -->
<!-- https://www.thasler.com/blog/blog/glsl-part2-emu -->
<!-- http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html -->
<!-- T. J. Dekker, A floating point technique for extending the available precision. -->
<!-- Numerische Mathematik, 18(3):224–242, 1971. -->
<!-- Andrew Thall, Extended-Precision Floating-Point Numbers for GPU Computation -->
<!-- SIGGRAPH, 2006 http://andrewthall.org/papers/df64_qf128.pdf -->
<!-- Single vs Double precision -->
<!-- ++++++++++++++++++++++++++ -->
<!-- Emulated arithmetics -->
<!-- ++++++++++++++++++++ -->
<!-- Emulated double precision array -->
<!-- +++++++++++++++++++++++++++++++ -->
</div>
</div>
<div class="section" id="id38">
<h2><a class="toc-backref" href="#id30">Conclusion</a></h2>
<p>As explained on the NumPy website, NumPy is the fundamental package for
scientific computing with Python. However, as illustrated in this chapter, the
usage of NumPy strengths goes far beyond a mere <em>multi-dimensional container of
generic data</em>. Using <code>ndarray</code> as a private property in one case (<code>TypedList</code>) or
directly subclassing the <code>ndarray</code> class (<code>GPUData</code>) to keep track of memory in
another case, we've seen how it is possible to extend NumPy's capabilities to
suit very specific needs. The limit is only your imagination and your experience.</p>
</div>
</div>
<div class="section" id="beyond-numpy">
<h1><a class="toc-backref" href="#table-of-contents">Beyond NumPy</a></h1>
<div class="contents local topic" id="id39">
<p class="topic-title first"><strong>Contents</strong></p>
<ul class="simple">
<li><a class="reference internal" href="#back-to-python" id="id177">Back to Python</a></li>
<li><a class="reference internal" href="#numpy-co" id="id178">NumPy &amp; co</a><ul>
<li><a class="reference internal" href="#numexpr" id="id179">NumExpr</a></li>
<li><a class="reference internal" href="#cython" id="id180">Cython</a></li>
<li><a class="reference internal" href="#numba" id="id181">Numba</a></li>
<li><a class="reference internal" href="#theano" id="id182">Theano</a></li>
<li><a class="reference internal" href="#pycuda" id="id183">PyCUDA</a></li>
<li><a class="reference internal" href="#pyopencl" id="id184">PyOpenCL</a></li>
</ul>
</li>
<li><a class="reference internal" href="#scipy-co" id="id185">Scipy &amp; co</a><ul>
<li><a class="reference internal" href="#scikit-learn" id="id186">scikit-learn</a></li>
<li><a class="reference internal" href="#scikit-image" id="id187">scikit-image</a></li>
<li><a class="reference internal" href="#sympy" id="id188">SymPy</a></li>
<li><a class="reference internal" href="#astropy" id="id189">Astropy</a></li>
<li><a class="reference internal" href="#cartopy" id="id190">Cartopy</a></li>
<li><a class="reference internal" href="#brian" id="id191">Brian</a></li>
<li><a class="reference internal" href="#id52" id="id192">Glumpy</a></li>
</ul>
</li>
<li><a class="reference internal" href="#id54" id="id193">Conclusion</a></li>
</ul>
</div>
<div class="section" id="back-to-python">
<h2><a class="toc-backref" href="#id39">Back to Python</a></h2>
<p>You've almost reached the end of the book and, hopefully, you've learned that
NumPy is a very versatile and powerful library. However in the meantime,
remember that Python is also quite a powerful language. In fact, in some
specific cases, it might be more powerful than NumPy. Let's consider, for
example, an interesting exercise that has been proposed by Tucker Balch in his
<a class="reference external" href="https://www.coursera.org/learn/computational-investing">Coursera's Computational Investing</a> course. The exercise
is written as:</p>
<blockquote>
<em>Write the most succinct code possible to compute all &quot;legal&quot; allocations to 4
stocks such that the allocations are in 1.0 chunks, and the allocations sum
to 10.0.</em></blockquote>
<p><a class="reference external" href="http://yasermartinez.com/blog/index.html">Yaser Martinez</a> collected the
different answers from the community and the proposed solutions yield
surprising results. But let's start with the most obvious Python solution:</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function">solution_1</span><span class="punctuation">():</span>
    <span class="comment single"># Brute force</span>
    <span class="comment single"># 14641 (=11*11*11*11) iterations &amp; tests</span>
    <span class="name">Z</span> <span class="operator">=</span> <span class="punctuation">[]</span>
    <span class="keyword">for</span> <span class="name">i</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="literal number integer">11</span><span class="punctuation">):</span>
        <span class="keyword">for</span> <span class="name">j</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="literal number integer">11</span><span class="punctuation">):</span>
            <span class="keyword">for</span> <span class="name">k</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="literal number integer">11</span><span class="punctuation">):</span>
                <span class="keyword">for</span> <span class="name">l</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="literal number integer">11</span><span class="punctuation">):</span>
                    <span class="keyword">if</span> <span class="name">i</span><span class="operator">+</span><span class="name">j</span><span class="operator">+</span><span class="name">k</span><span class="operator">+</span><span class="name">l</span> <span class="operator">==</span> <span class="literal number integer">10</span><span class="punctuation">:</span>
                        <span class="name">Z</span><span class="operator">.</span><span class="name">append</span><span class="punctuation">((</span><span class="name">i</span><span class="punctuation">,</span><span class="name">j</span><span class="punctuation">,</span><span class="name">k</span><span class="punctuation">,</span><span class="name">l</span><span class="punctuation">))</span>
    <span class="keyword">return</span> <span class="name">Z</span>
</pre>
<p>This solution is the slowest solution because it requires 4 loops, and more
importantly, it tests all the different combinations (11641) of 4 integers
between 0 and 10 to retain only combinations whose sum is 10. We can of course
get rid of the 4 loops using itertools, but the code remains slow:</p>
<pre class="code python literal-block">
<span class="keyword namespace">import</span> <span class="name namespace">itertools</span> <span class="keyword namespace">as</span> <span class="name namespace">it</span>

<span class="keyword">def</span> <span class="name function">solution_2</span><span class="punctuation">():</span>
    <span class="comment single"># Itertools</span>
    <span class="comment single"># 14641 (=11*11*11*11) iterations &amp; tests</span>
    <span class="keyword">return</span> <span class="punctuation">[(</span><span class="name">i</span><span class="punctuation">,</span><span class="name">j</span><span class="punctuation">,</span><span class="name">k</span><span class="punctuation">,</span><span class="name">l</span><span class="punctuation">)</span>
            <span class="keyword">for</span> <span class="name">i</span><span class="punctuation">,</span><span class="name">j</span><span class="punctuation">,</span><span class="name">k</span><span class="punctuation">,</span><span class="name">l</span> <span class="operator word">in</span> <span class="name">it</span><span class="operator">.</span><span class="name">product</span><span class="punctuation">(</span><span class="name builtin">range</span><span class="punctuation">(</span><span class="literal number integer">11</span><span class="punctuation">),</span><span class="name">repeat</span><span class="operator">=</span><span class="literal number integer">4</span><span class="punctuation">)</span> <span class="keyword">if</span> <span class="name">i</span><span class="operator">+</span><span class="name">j</span><span class="operator">+</span><span class="name">k</span><span class="operator">+</span><span class="name">l</span> <span class="operator">==</span> <span class="literal number integer">10</span><span class="punctuation">]</span>
</pre>
<p>One of the best solution that has been proposed by Nick Popplas takes advantage
of the fact we can have intelligent imbricated loops that will allow us to
directly build each tuple without any test as shown below:</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function">solution_3</span><span class="punctuation">():</span>
    <span class="keyword">return</span> <span class="punctuation">[(</span><span class="name">a</span><span class="punctuation">,</span> <span class="name">b</span><span class="punctuation">,</span> <span class="name">c</span><span class="punctuation">,</span> <span class="punctuation">(</span><span class="literal number integer">10</span> <span class="operator">-</span> <span class="name">a</span> <span class="operator">-</span> <span class="name">b</span> <span class="operator">-</span> <span class="name">c</span><span class="punctuation">))</span>
            <span class="keyword">for</span> <span class="name">a</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="literal number integer">11</span><span class="punctuation">)</span>
            <span class="keyword">for</span> <span class="name">b</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="literal number integer">11</span> <span class="operator">-</span> <span class="name">a</span><span class="punctuation">)</span>
            <span class="keyword">for</span> <span class="name">c</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="literal number integer">11</span> <span class="operator">-</span> <span class="name">a</span> <span class="operator">-</span> <span class="name">b</span><span class="punctuation">)]</span>
</pre>
<p>The best NumPy solution by Yaser Martinez uses a different strategy with a
restricted set of tests:</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function">solution_4</span><span class="punctuation">():</span>
    <span class="name">X123</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">indices</span><span class="punctuation">((</span><span class="literal number integer">11</span><span class="punctuation">,</span><span class="literal number integer">11</span><span class="punctuation">,</span><span class="literal number integer">11</span><span class="punctuation">))</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="literal number integer">3</span><span class="punctuation">,</span><span class="literal number integer">11</span><span class="operator">*</span><span class="literal number integer">11</span><span class="operator">*</span><span class="literal number integer">11</span><span class="punctuation">)</span>
    <span class="name">X4</span> <span class="operator">=</span> <span class="literal number integer">10</span> <span class="operator">-</span> <span class="name">X123</span><span class="operator">.</span><span class="name">sum</span><span class="punctuation">(</span><span class="name">axis</span><span class="operator">=</span><span class="literal number integer">0</span><span class="punctuation">)</span>
    <span class="keyword">return</span> <span class="name">np</span><span class="operator">.</span><span class="name">vstack</span><span class="punctuation">((</span><span class="name">X123</span><span class="punctuation">,</span> <span class="name">X4</span><span class="punctuation">))</span><span class="operator">.</span><span class="name">T</span><span class="punctuation">[</span><span class="name">X4</span> <span class="operator">&gt;</span> <span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">]</span>
</pre>
<p>If we benchmark these methods, we get:</p>
<pre class="code pycon literal-block">
<span class="name"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;solution_1()&quot;</span><span class="punctuation">,</span> <span class="name builtin">globals</span><span class="punctuation">())</span>
<span class="generic output">100 loops, best of 3: 1.9 msec per loop
</span><span class="name"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;solution_2()&quot;</span><span class="punctuation">,</span> <span class="name builtin">globals</span><span class="punctuation">())</span>
<span class="generic output">100 loops, best of 3: 1.67 msec per loop
</span><span class="name"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;solution_3()&quot;</span><span class="punctuation">,</span> <span class="name builtin">globals</span><span class="punctuation">())</span>
<span class="generic output">1000 loops, best of 3: 60.4 usec per loop
</span><span class="name"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;solution_4()&quot;</span><span class="punctuation">,</span> <span class="name builtin">globals</span><span class="punctuation">())</span>
<span class="generic output">1000 loops, best of 3: 54.4 usec per loop</span>
</pre>
<p>The NumPy solution is the fastest but the pure Python solution is comparable.
But let me introduce a small modification to the Python solution:</p>
<pre class="code python literal-block">
<span class="keyword">def</span> <span class="name function">solution_3_bis</span><span class="punctuation">():</span>
    <span class="keyword">return</span> <span class="punctuation">((</span><span class="name">a</span><span class="punctuation">,</span> <span class="name">b</span><span class="punctuation">,</span> <span class="name">c</span><span class="punctuation">,</span> <span class="punctuation">(</span><span class="literal number integer">10</span> <span class="operator">-</span> <span class="name">a</span> <span class="operator">-</span> <span class="name">b</span> <span class="operator">-</span> <span class="name">c</span><span class="punctuation">))</span>
            <span class="keyword">for</span> <span class="name">a</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="literal number integer">11</span><span class="punctuation">)</span>
            <span class="keyword">for</span> <span class="name">b</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="literal number integer">11</span> <span class="operator">-</span> <span class="name">a</span><span class="punctuation">)</span>
            <span class="keyword">for</span> <span class="name">c</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="literal number integer">11</span> <span class="operator">-</span> <span class="name">a</span> <span class="operator">-</span> <span class="name">b</span><span class="punctuation">))</span>
</pre>
<p>If we benchmark it, we get:</p>
<pre class="code pycon literal-block">
<span class="name"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="name">timeit</span><span class="punctuation">(</span><span class="literal string double">&quot;solution_3_bis()&quot;</span><span class="punctuation">,</span> <span class="name builtin">globals</span><span class="punctuation">())</span>
<span class="generic output">10000 loops, best of 3: 0.643 usec per loop</span>
</pre>
<p>You read that right, we have gained a factor of 100 just by replacing square
brackets with parenthesis. How is that possible? The explanation can be found
by looking at the type of the returned object:</p>
<pre class="code pycon literal-block">
<span class="keyword"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="keyword">print</span><span class="punctuation">(</span><span class="name builtin">type</span><span class="punctuation">(</span><span class="name">solution_3</span><span class="punctuation">()))</span>
<span class="generic output">&lt;class 'list'&gt;
</span><span class="keyword"></span><span class="generic prompt">&gt;&gt;&gt; </span><span class="keyword">print</span><span class="punctuation">(</span><span class="name builtin">type</span><span class="punctuation">(</span><span class="name">solution_3_bis</span><span class="punctuation">()))</span>
<span class="generic output">&lt;class 'generator'&gt;</span>
</pre>
<p>The <code>solution_3_bis()</code> returns a generator that can be used to generate the
full list or to iterate over all the different elements. In any case, the huge
speedup comes from the non-instantiation of the full list and it is thus
important to wonder if you need an actual instance of your result or if a
simple generator might do the job.</p>
</div>
<div class="section" id="numpy-co">
<h2><a class="toc-backref" href="#id39">NumPy &amp; co</a></h2>
<p>Beyond NumPy, there are several other Python packages that are worth a look
because they address similar yet different class of problems using different
technology (compilation, virtual machine, just in time compilation, GPU,
compression, etc.). Depending on your specific problem and your hardware, one
package may be better than the other. Let's illustrate their usage using a very
simple example where we want to compute an expression based on two float
vectors:</p>
<pre class="code python literal-block">
<span class="keyword namespace">import</span> <span class="name namespace">numpy</span> <span class="keyword namespace">as</span> <span class="name namespace">np</span>
<span class="name">a</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">random</span><span class="operator">.</span><span class="name">uniform</span><span class="punctuation">(</span><span class="literal number integer">0</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">,</span> <span class="literal number integer">1000</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">astype</span><span class="punctuation">(</span><span class="name">np</span><span class="operator">.</span><span class="name">float32</span><span class="punctuation">)</span>
<span class="name">b</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">random</span><span class="operator">.</span><span class="name">uniform</span><span class="punctuation">(</span><span class="literal number integer">0</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">,</span> <span class="literal number integer">1000</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">astype</span><span class="punctuation">(</span><span class="name">np</span><span class="operator">.</span><span class="name">float32</span><span class="punctuation">)</span>
<span class="name">c</span> <span class="operator">=</span> <span class="literal number integer">2</span><span class="operator">*</span><span class="name">a</span> <span class="operator">+</span> <span class="literal number integer">3</span><span class="operator">*</span><span class="name">b</span>
</pre>
<div class="section" id="numexpr">
<h3><a class="toc-backref" href="#id39">NumExpr</a></h3>
<p>The <a class="reference external" href="https://github.com/pydata/numexpr/wiki/Numexpr-Users-Guide">numexpr</a>
package supplies routines for the fast evaluation of array expressions
element-wise by using a vector-based virtual machine. It's comparable to SciPy's
weave package, but doesn't require a separate compile step of C or C++ code.</p>
<pre class="code python literal-block">
<span class="keyword namespace">import</span> <span class="name namespace">numpy</span> <span class="keyword namespace">as</span> <span class="name namespace">np</span>
<span class="keyword namespace">import</span> <span class="name namespace">numexpr</span> <span class="keyword namespace">as</span> <span class="name namespace">ne</span>

<span class="name">a</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">random</span><span class="operator">.</span><span class="name">uniform</span><span class="punctuation">(</span><span class="literal number integer">0</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">,</span> <span class="literal number integer">1000</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">astype</span><span class="punctuation">(</span><span class="name">np</span><span class="operator">.</span><span class="name">float32</span><span class="punctuation">)</span>
<span class="name">b</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">random</span><span class="operator">.</span><span class="name">uniform</span><span class="punctuation">(</span><span class="literal number integer">0</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">,</span> <span class="literal number integer">1000</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">astype</span><span class="punctuation">(</span><span class="name">np</span><span class="operator">.</span><span class="name">float32</span><span class="punctuation">)</span>
<span class="name">c</span> <span class="operator">=</span> <span class="name">ne</span><span class="operator">.</span><span class="name">evaluate</span><span class="punctuation">(</span><span class="literal string double">&quot;2*a + 3*b&quot;</span><span class="punctuation">)</span>
</pre>
</div>
<div class="section" id="cython">
<h3><a class="toc-backref" href="#id39">Cython</a></h3>
<p><a class="reference external" href="http://cython.org">Cython</a> is an optimising static compiler for both the
Python programming language and the extended Cython programming language (based
on Pyrex). It makes writing C extensions for Python as easy as Python itself.</p>
<pre class="code python literal-block">
<span class="keyword namespace">import</span> <span class="name namespace">numpy</span> <span class="keyword namespace">as</span> <span class="name namespace">np</span>

<span class="keyword">def</span> <span class="name function">evaluate</span><span class="punctuation">(</span><span class="name">np</span><span class="operator">.</span><span class="name">ndarray</span> <span class="name">a</span><span class="punctuation">,</span> <span class="name">np</span><span class="operator">.</span><span class="name">ndarray</span> <span class="name">b</span><span class="punctuation">):</span>
    <span class="name">cdef</span> <span class="name builtin">int</span> <span class="name">i</span>
    <span class="name">cdef</span> <span class="name">np</span><span class="operator">.</span><span class="name">ndarray</span> <span class="name">c</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">zeros_like</span><span class="punctuation">(</span><span class="name">a</span><span class="punctuation">)</span>
    <span class="keyword">for</span> <span class="name">i</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="name">a</span><span class="operator">.</span><span class="name">size</span><span class="punctuation">):</span>
        <span class="name">c</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="literal number integer">2</span><span class="operator">*</span><span class="name">a</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">]</span> <span class="operator">+</span> <span class="literal number integer">3</span><span class="operator">*</span><span class="name">b</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">]</span>
    <span class="keyword">return</span> <span class="name">c</span>

<span class="name">a</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">random</span><span class="operator">.</span><span class="name">uniform</span><span class="punctuation">(</span><span class="literal number integer">0</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">,</span> <span class="literal number integer">1000</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">astype</span><span class="punctuation">(</span><span class="name">np</span><span class="operator">.</span><span class="name">float32</span><span class="punctuation">)</span>
<span class="name">b</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">random</span><span class="operator">.</span><span class="name">uniform</span><span class="punctuation">(</span><span class="literal number integer">0</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">,</span> <span class="literal number integer">1000</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">astype</span><span class="punctuation">(</span><span class="name">np</span><span class="operator">.</span><span class="name">float32</span><span class="punctuation">)</span>
<span class="name">c</span> <span class="operator">=</span> <span class="name">evaluate</span><span class="punctuation">(</span><span class="name">a</span><span class="punctuation">,</span> <span class="name">b</span><span class="punctuation">)</span>
</pre>
</div>
<div class="section" id="numba">
<h3><a class="toc-backref" href="#id39">Numba</a></h3>
<p><a class="reference external" href="http://numba.pydata.org">Numba</a> gives you the power to speed up your
applications with high performance functions written directly in Python. With a
few annotations, array-oriented and math-heavy Python code can be just-in-time
compiled to native machine instructions, similar in performance to C, C++ and
Fortran, without having to switch languages or Python interpreters.</p>
<pre class="code python literal-block">
<span class="keyword namespace">from</span> <span class="name namespace">numba</span> <span class="keyword namespace">import</span> <span class="name">jit</span>
<span class="keyword namespace">import</span> <span class="name namespace">numpy</span> <span class="keyword namespace">as</span> <span class="name namespace">np</span>

<span class="name decorator">&#64;jit</span>
<span class="keyword">def</span> <span class="name function">evaluate</span><span class="punctuation">(</span><span class="name">np</span><span class="operator">.</span><span class="name">ndarray</span> <span class="name">a</span><span class="punctuation">,</span> <span class="name">np</span><span class="operator">.</span><span class="name">ndarray</span> <span class="name">b</span><span class="punctuation">):</span>
    <span class="name">c</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">zeros_like</span><span class="punctuation">(</span><span class="name">a</span><span class="punctuation">)</span>
    <span class="keyword">for</span> <span class="name">i</span> <span class="operator word">in</span> <span class="name builtin">range</span><span class="punctuation">(</span><span class="name">a</span><span class="operator">.</span><span class="name">size</span><span class="punctuation">):</span>
        <span class="name">c</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">]</span> <span class="operator">=</span> <span class="literal number integer">2</span><span class="operator">*</span><span class="name">a</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">]</span> <span class="operator">+</span> <span class="literal number integer">3</span><span class="operator">*</span><span class="name">b</span><span class="punctuation">[</span><span class="name">i</span><span class="punctuation">]</span>
    <span class="keyword">return</span> <span class="name">c</span>

<span class="name">a</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">random</span><span class="operator">.</span><span class="name">uniform</span><span class="punctuation">(</span><span class="literal number integer">0</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">,</span> <span class="literal number integer">1000</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">astype</span><span class="punctuation">(</span><span class="name">np</span><span class="operator">.</span><span class="name">float32</span><span class="punctuation">)</span>
<span class="name">b</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">random</span><span class="operator">.</span><span class="name">uniform</span><span class="punctuation">(</span><span class="literal number integer">0</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">,</span> <span class="literal number integer">1000</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">astype</span><span class="punctuation">(</span><span class="name">np</span><span class="operator">.</span><span class="name">float32</span><span class="punctuation">)</span>
<span class="name">c</span> <span class="operator">=</span> <span class="name">evaluate</span><span class="punctuation">(</span><span class="name">a</span><span class="punctuation">,</span> <span class="name">b</span><span class="punctuation">)</span>
</pre>
</div>
<div class="section" id="theano">
<h3><a class="toc-backref" href="#id39">Theano</a></h3>
<p><a class="reference external" href="http://www.deeplearning.net/software/theano/">Theano</a> is a Python library
that allows you to define, optimize, and evaluate mathematical expressions
involving multi-dimensional arrays efficiently. Theano features tight
integration with NumPy, transparent use of a GPU, efficient symbolic
differentiation, speed and stability optimizations, dynamic C code generation
and extensive unit-testing and self-verification.</p>
<pre class="code python literal-block">
<span class="keyword namespace">import</span> <span class="name namespace">numpy</span> <span class="keyword namespace">as</span> <span class="name namespace">np</span>
<span class="keyword namespace">import</span> <span class="name namespace">theano.tensor</span> <span class="keyword namespace">as</span> <span class="name namespace">T</span>

<span class="name">x</span> <span class="operator">=</span> <span class="name">T</span><span class="operator">.</span><span class="name">fvector</span><span class="punctuation">(</span><span class="literal string single">'x'</span><span class="punctuation">)</span>
<span class="name">y</span> <span class="operator">=</span> <span class="name">T</span><span class="operator">.</span><span class="name">fvector</span><span class="punctuation">(</span><span class="literal string single">'y'</span><span class="punctuation">)</span>
<span class="name">z</span> <span class="operator">=</span> <span class="literal number integer">2</span><span class="operator">*</span><span class="name">x</span> <span class="operator">+</span> <span class="literal number integer">3</span><span class="operator">*</span><span class="name">y</span>
<span class="name">f</span> <span class="operator">=</span> <span class="name">function</span><span class="punctuation">([</span><span class="name">x</span><span class="punctuation">,</span> <span class="name">y</span><span class="punctuation">],</span> <span class="name">z</span><span class="punctuation">)</span>

<span class="name">a</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">random</span><span class="operator">.</span><span class="name">uniform</span><span class="punctuation">(</span><span class="literal number integer">0</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">,</span> <span class="literal number integer">1000</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">astype</span><span class="punctuation">(</span><span class="name">np</span><span class="operator">.</span><span class="name">float32</span><span class="punctuation">)</span>
<span class="name">b</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">random</span><span class="operator">.</span><span class="name">uniform</span><span class="punctuation">(</span><span class="literal number integer">0</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">,</span> <span class="literal number integer">1000</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">astype</span><span class="punctuation">(</span><span class="name">np</span><span class="operator">.</span><span class="name">float32</span><span class="punctuation">)</span>
<span class="name">c</span> <span class="operator">=</span> <span class="name">f</span><span class="punctuation">(</span><span class="name">a</span><span class="punctuation">,</span> <span class="name">b</span><span class="punctuation">)</span>
</pre>
</div>
<div class="section" id="pycuda">
<h3><a class="toc-backref" href="#id39">PyCUDA</a></h3>
<p><a class="reference external" href="http://mathema.tician.de/software/pycuda">PyCUDA</a> lets you access Nvidia's
CUDA parallel computation API from Python.</p>
<pre class="code python literal-block">
<span class="keyword namespace">import</span> <span class="name namespace">numpy</span> <span class="keyword namespace">as</span> <span class="name namespace">np</span>
<span class="keyword namespace">import</span> <span class="name namespace">pycuda.autoinit</span>
<span class="keyword namespace">import</span> <span class="name namespace">pycuda.driver</span> <span class="keyword namespace">as</span> <span class="name namespace">drv</span>
<span class="keyword namespace">from</span> <span class="name namespace">pycuda.compiler</span> <span class="keyword namespace">import</span> <span class="name">SourceModule</span>

<span class="name">mod</span> <span class="operator">=</span> <span class="name">SourceModule</span><span class="punctuation">(</span><span class="literal string double">&quot;&quot;&quot;
    __global__ void evaluate(float *c, float *a, float *b)
    {
      const int i = threadIdx.x;
      c[i] = 2*a[i] + 3*b[i];
    }
&quot;&quot;&quot;</span><span class="punctuation">)</span>

<span class="name">evaluate</span> <span class="operator">=</span> <span class="name">mod</span><span class="operator">.</span><span class="name">get_function</span><span class="punctuation">(</span><span class="literal string double">&quot;evaluate&quot;</span><span class="punctuation">)</span>

<span class="name">a</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">random</span><span class="operator">.</span><span class="name">uniform</span><span class="punctuation">(</span><span class="literal number integer">0</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">,</span> <span class="literal number integer">1000</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">astype</span><span class="punctuation">(</span><span class="name">np</span><span class="operator">.</span><span class="name">float32</span><span class="punctuation">)</span>
<span class="name">b</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">random</span><span class="operator">.</span><span class="name">uniform</span><span class="punctuation">(</span><span class="literal number integer">0</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">,</span> <span class="literal number integer">1000</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">astype</span><span class="punctuation">(</span><span class="name">np</span><span class="operator">.</span><span class="name">float32</span><span class="punctuation">)</span>
<span class="name">c</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">zeros_like</span><span class="punctuation">(</span><span class="name">a</span><span class="punctuation">)</span>

<span class="name">evaluate</span><span class="punctuation">(</span><span class="name">drv</span><span class="operator">.</span><span class="name">Out</span><span class="punctuation">(</span><span class="name">c</span><span class="punctuation">),</span> <span class="name">drv</span><span class="operator">.</span><span class="name">In</span><span class="punctuation">(</span><span class="name">a</span><span class="punctuation">),</span> <span class="name">drv</span><span class="operator">.</span><span class="name">In</span><span class="punctuation">(</span><span class="name">b</span><span class="punctuation">),</span> <span class="name">block</span><span class="operator">=</span><span class="punctuation">(</span><span class="literal number integer">400</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">),</span> <span class="name">grid</span><span class="operator">=</span><span class="punctuation">(</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">))</span>
</pre>
</div>
<div class="section" id="pyopencl">
<h3><a class="toc-backref" href="#id39">PyOpenCL</a></h3>
<p><a class="reference external" href="http://mathema.tician.de/software/pyopencl">PyOpenCL</a> lets you access GPUs
and other massively parallel compute devices from Python.</p>
<pre class="code python literal-block">
<span class="keyword namespace">import</span> <span class="name namespace">numpy</span> <span class="keyword namespace">as</span> <span class="name namespace">np</span>
<span class="keyword namespace">import</span> <span class="name namespace">pyopencl</span> <span class="keyword namespace">as</span> <span class="name namespace">cl</span>

<span class="name">a</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">random</span><span class="operator">.</span><span class="name">uniform</span><span class="punctuation">(</span><span class="literal number integer">0</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">,</span> <span class="literal number integer">1000</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">astype</span><span class="punctuation">(</span><span class="name">np</span><span class="operator">.</span><span class="name">float32</span><span class="punctuation">)</span>
<span class="name">b</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">random</span><span class="operator">.</span><span class="name">uniform</span><span class="punctuation">(</span><span class="literal number integer">0</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">,</span> <span class="literal number integer">1000</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">astype</span><span class="punctuation">(</span><span class="name">np</span><span class="operator">.</span><span class="name">float32</span><span class="punctuation">)</span>
<span class="name">c</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">empty_like</span><span class="punctuation">(</span><span class="name">a</span><span class="punctuation">)</span>

<span class="name">ctx</span> <span class="operator">=</span> <span class="name">cl</span><span class="operator">.</span><span class="name">create_some_context</span><span class="punctuation">()</span>
<span class="name">queue</span> <span class="operator">=</span> <span class="name">cl</span><span class="operator">.</span><span class="name">CommandQueue</span><span class="punctuation">(</span><span class="name">ctx</span><span class="punctuation">)</span>

<span class="name">mf</span> <span class="operator">=</span> <span class="name">cl</span><span class="operator">.</span><span class="name">mem_flags</span>
<span class="name">gpu_a</span> <span class="operator">=</span> <span class="name">cl</span><span class="operator">.</span><span class="name">Buffer</span><span class="punctuation">(</span><span class="name">ctx</span><span class="punctuation">,</span> <span class="name">mf</span><span class="operator">.</span><span class="name">READ_ONLY</span> <span class="operator">|</span> <span class="name">mf</span><span class="operator">.</span><span class="name">COPY_HOST_PTR</span><span class="punctuation">,</span> <span class="name">hostbuf</span><span class="operator">=</span><span class="name">a</span><span class="punctuation">)</span>
<span class="name">gpu_b</span> <span class="operator">=</span> <span class="name">cl</span><span class="operator">.</span><span class="name">Buffer</span><span class="punctuation">(</span><span class="name">ctx</span><span class="punctuation">,</span> <span class="name">mf</span><span class="operator">.</span><span class="name">READ_ONLY</span> <span class="operator">|</span> <span class="name">mf</span><span class="operator">.</span><span class="name">COPY_HOST_PTR</span><span class="punctuation">,</span> <span class="name">hostbuf</span><span class="operator">=</span><span class="name">b</span><span class="punctuation">)</span>

<span class="name">evaluate</span> <span class="operator">=</span> <span class="name">cl</span><span class="operator">.</span><span class="name">Program</span><span class="punctuation">(</span><span class="name">ctx</span><span class="punctuation">,</span> <span class="literal string double">&quot;&quot;&quot;
    __kernel void evaluate(__global const float *gpu_a;
                           __global const float *gpu_b;
                           __global       float *gpu_c)
    {
        int gid = get_global_id(0);
        gpu_c[gid] = 2*gpu_a[gid] + 3*gpu_b[gid];
    }
&quot;&quot;&quot;</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">build</span><span class="punctuation">()</span>

<span class="name">gpu_c</span> <span class="operator">=</span> <span class="name">cl</span><span class="operator">.</span><span class="name">Buffer</span><span class="punctuation">(</span><span class="name">ctx</span><span class="punctuation">,</span> <span class="name">mf</span><span class="operator">.</span><span class="name">WRITE_ONLY</span><span class="punctuation">,</span> <span class="name">a</span><span class="operator">.</span><span class="name">nbytes</span><span class="punctuation">)</span>
<span class="name">evaluate</span><span class="operator">.</span><span class="name">evaluate</span><span class="punctuation">(</span><span class="name">queue</span><span class="punctuation">,</span> <span class="name">a</span><span class="operator">.</span><span class="name">shape</span><span class="punctuation">,</span> <span class="name builtin pseudo">None</span><span class="punctuation">,</span> <span class="name">gpu_a</span><span class="punctuation">,</span> <span class="name">gpu_b</span><span class="punctuation">,</span> <span class="name">gpu_c</span><span class="punctuation">)</span>
<span class="name">cl</span><span class="operator">.</span><span class="name">enqueue_copy</span><span class="punctuation">(</span><span class="name">queue</span><span class="punctuation">,</span> <span class="name">c</span><span class="punctuation">,</span> <span class="name">gpu_c</span><span class="punctuation">)</span>
</pre>
</div>
</div>
<div class="section" id="scipy-co">
<h2><a class="toc-backref" href="#id39">Scipy &amp; co</a></h2>
<p>If there are several additional packages for NumPy, there are a trillion
additional packages for scipy. In fact, every domain of science probably has
its own package and most of the examples we've been studying until now could
have been solved in two or three calls to a method in the relevant package.
But of course, that was not the goal and programming things yourself is generally
a good exercise if you have some spare time. The biggest difficulty at this
point is to find these relevant packages. Here is a very short list of packages
that are well-maintained, well-tested and may simplify your scientific life
(depending on your domain). There are of course many more and depending on your
specific needs, chances are you do not have to program everything by
yourself. For an extensive list, have a look at the <a class="reference external" href="https://awesome-python.com">Awesome python list</a>.</p>
<div class="section" id="scikit-learn">
<h3><a class="toc-backref" href="#id39">scikit-learn</a></h3>
<p><a class="reference external" href="http://scikit-learn.org/stable/">scikit-learn</a> is a free software machine
learning library for the Python programming language. It features various
classification, regression and clustering algorithms including support vector
machines, random forests, gradient boosting, k-means and DBSCAN, and is
designed to inter-operate with the Python numerical and scientific libraries
NumPy and SciPy.</p>
</div>
<div class="section" id="scikit-image">
<h3><a class="toc-backref" href="#id39">scikit-image</a></h3>
<p><a class="reference external" href="http://scikit-image.org">scikit-image</a> is a Python package dedicated to
image processing, and using natively NumPy arrays as image objects. This
chapter describes how to use scikit-image on various image processing tasks,
and insists on the link with other scientific Python modules such as NumPy and
SciPy.</p>
</div>
<div class="section" id="sympy">
<h3><a class="toc-backref" href="#id39">SymPy</a></h3>
<p><a class="reference external" href="http://www.sympy.org/en/index.html">SymPy</a> is a Python library for symbolic
mathematics. It aims to become a full-featured computer algebra system (CAS)
while keeping the code as simple as possible in order to be comprehensible and
easily extensible. SymPy is written entirely in Python.</p>
</div>
<div class="section" id="astropy">
<h3><a class="toc-backref" href="#id39">Astropy</a></h3>
<p>The <a class="reference external" href="http://www.astropy.org">Astropy</a> project is a community effort to
develop a single core package for astronomy in Python and foster
interoperability between Python astronomy packages.</p>
</div>
<div class="section" id="cartopy">
<h3><a class="toc-backref" href="#id39">Cartopy</a></h3>
<p><a class="reference external" href="http://scitools.org.uk/cartopy/">Cartopy</a> is a Python package designed to
make drawing maps for data analysis and visualization as easy as
possible. Cartopy makes use of the powerful PROJ.4, NumPy and shapely libraries
and has a simple and intuitive drawing interface to matplotlib for creating
publication quality maps.</p>
</div>
<div class="section" id="brian">
<h3><a class="toc-backref" href="#id39">Brian</a></h3>
<p><a class="reference external" href="http://www.briansimulator.org">Brian</a> is a free, open source simulator for
spiking neural networks. It is written in the Python programming language and
is available on almost all platforms. We believe that a simulator should not
only save the time of processors, but also the time of scientists. Brian is
therefore designed to be easy to learn and use, highly flexible and easily
extensible.</p>
</div>
<div class="section" id="id52">
<h3><a class="toc-backref" href="#id39">Glumpy</a></h3>
<p><a class="reference external" href="http://glumpy.github.io">Glumpy</a> is an OpenGL-based interactive
visualization library in Python. Its goal is to make it easy to create fast,
scalable, beautiful, interactive and dynamic visualizations. The main
documentation for the site is organized into a couple of sections:</p>
</div>
</div>
<div class="section" id="id54">
<h2><a class="toc-backref" href="#id39">Conclusion</a></h2>
<p>NumPy is a very versatile library but still, it does not mean you have to use
it in every situation. In this chapter, we've seen some alternatives (including
Python itself) that are worth a look. As always, the choice belongs to you. You
have to consider what is the best solution for you in term of development time,
computation time and effort in maintenance. On the one hand, if you design your
own solution, you'll have to test it and to maintain it, but in exchange,
you'll be free to design it the way you want. On the other hand, if you decide
to rely on a third-party package, you'll save time in development and benefit
from community-support even though you might have to adapt the package to your
specific needs. The choice is up to you.</p>
</div>
</div>
<div class="section" id="id55">
<h1><a class="toc-backref" href="#table-of-contents">Conclusion</a></h1>
<p>You've reached the end of this book. I hope you've learned something while
reading it, I sure learned a lot writing it. Trying to explain something is a
generally a good exercise to test for your knowledge of this thing. Of course,
we only scratched the surface of NumPy and there are many things left to
discover. Have a look at the bibliography for books written by true experts, at
the documentation written by people making NumPy and don't hesitate to ask your
questions on the mailing lists because the NumPy community is very friendly.</p>
<p>If there's a single message to retain from this book it is &quot;premature
optimization is the root of all evil&quot;. We've seen that code vectorization can
drastically improve your computation, with several orders of magnitude in some
cases. Still, problem vectorization is generally much more powerful. If you
write code vectorization too early in your design process, you won't be able to
think out-of-the-box and you'll certainly miss some really powerful alternatives
because you won't be able to identify your problem properly as we've
seen in the problem vectorization chapter. This requires some experience and
you have to be patient: experience is not an overnight process.</p>
<p>Finally, custom vectorization is an option worth considering once you've looked
at the alternatives to NumPy. When nothing works for you, NumPy still offers
you a clever framework to forge your own tools. And who knows, this can be the
start of an exciting adventure for you and the community as it happened to me
with the <a class="reference external" href="http://glumpy.github.io">glumpy</a> and the <a class="reference external" href="http://vispy.org">vispy</a> packages.</p>
</div>
<div class="section" id="quick-references">
<h1><a class="toc-backref" href="#table-of-contents">Quick References</a></h1>
<div class="contents local topic" id="id57">
<p class="topic-title first"><strong>Contents</strong></p>
<ul class="simple">
<li><a class="reference internal" href="#id58" id="id194">Data type</a></li>
<li><a class="reference internal" href="#id59" id="id195">Creation</a></li>
<li><a class="reference internal" href="#id60" id="id196">Indexing</a></li>
<li><a class="reference internal" href="#reshaping" id="id197">Reshaping</a></li>
<li><a class="reference internal" href="#broadcasting" id="id198">Broadcasting</a></li>
</ul>
</div>
<div class="section" id="id58">
<h2><a class="toc-backref" href="#id57">Data type</a></h2>
<table border="1" class="docutils">
<colgroup>
<col width="16%" />
<col width="7%" />
<col width="6%" />
<col width="71%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">Type</th>
<th class="head">Name</th>
<th class="head">Bytes</th>
<th class="head">Description</th>
</tr>
</thead>
<tbody valign="top">
<tr><td><code>bool</code></td>
<td><code>b</code></td>
<td>1</td>
<td>Boolean (True or False) stored as a byte</td>
</tr>
<tr><td><code>int</code></td>
<td><code>l</code></td>
<td>4-8</td>
<td>Platform (long) integer (normally either int32 or int64)</td>
</tr>
<tr><td><code>intp</code></td>
<td><code>p</code></td>
<td>4-8</td>
<td>Integer used for indexing (normally either int32 or int64)</td>
</tr>
<tr><td><code>int8</code></td>
<td><code>i1</code></td>
<td>1</td>
<td>Byte (-128 to 127)</td>
</tr>
<tr><td><code>int16</code></td>
<td><code>i2</code></td>
<td>2</td>
<td>Integer (-32768 to 32767)</td>
</tr>
<tr><td><code>int32</code></td>
<td><code>i4</code></td>
<td>4</td>
<td>Integer (-2147483648 to 2147483647)</td>
</tr>
<tr><td><code>int64</code></td>
<td><code>i8</code></td>
<td>8</td>
<td>Integer (-9223372036854775808 to 9223372036854775807)</td>
</tr>
<tr><td><code>uint8</code></td>
<td><code>u1</code></td>
<td>1</td>
<td>Unsigned integer (0 to 255)</td>
</tr>
<tr><td><code>uint16</code></td>
<td><code>u2</code></td>
<td>2</td>
<td>Unsigned integer (0 to 65535)</td>
</tr>
<tr><td><code>uint32</code></td>
<td><code>u4</code></td>
<td>4</td>
<td>Unsigned integer (0 to 4294967295)</td>
</tr>
<tr><td><code>uint64</code></td>
<td><code>u8</code></td>
<td>8</td>
<td>Unsigned integer (0 to 18446744073709551615)</td>
</tr>
<tr><td><code>float</code></td>
<td><code>f8</code></td>
<td>8</td>
<td>Shorthand for float64</td>
</tr>
<tr><td><code>float16</code></td>
<td><code>f2</code></td>
<td>2</td>
<td>Half precision float:
sign bit, 5 bits exponent, 10 bits mantissa</td>
</tr>
<tr><td><code>float32</code></td>
<td><code>f</code></td>
<td>4</td>
<td>Single precision float:
sign bit, 8 bits exponent, 23 bits mantissa</td>
</tr>
<tr><td><code>float64</code></td>
<td><code>d</code></td>
<td>8</td>
<td>Double precision float:
sign bit, 11 bits exponent, 52 bits mantissa</td>
</tr>
<tr><td><code>complex</code></td>
<td><code>c16</code></td>
<td>16</td>
<td>Shorthand for complex128.</td>
</tr>
<tr><td><code>complex64</code></td>
<td><code>c8</code></td>
<td>8</td>
<td>Complex number, represented by two 32-bit floats</td>
</tr>
<tr><td><code>complex128</code></td>
<td><code>c16</code></td>
<td>16</td>
<td>Complex number, represented by two 64-bit floats</td>
</tr>
</tbody>
</table>
<p><code>bool</code>, <code>int</code>, <code>float</code>, and <code>complex</code> are understood, but named <code>np.bool_</code> with
an additional underscore in NumPy. Additionally the names such as <code>intc</code>,
<code>long</code>, or <code>double</code>  used in the C programming language are defined.</p>
</div>
<div class="section" id="id59">
<h2><a class="toc-backref" href="#id57">Creation</a></h2>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<pre class="code python literal-block">
<span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">zeros</span><span class="punctuation">(</span><span class="literal number integer">9</span><span class="punctuation">)</span>
</pre>
<pre class="code output literal-block">
┌───┬───┬───┬───┬───┬───┬───┬───┬───┐
│ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │
└───┴───┴───┴───┴───┴───┴───┴───┴───┘
</pre>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<pre class="code python literal-block">
<span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">ones</span><span class="punctuation">(</span><span class="literal number integer">9</span><span class="punctuation">)</span>
</pre>
<pre class="code output literal-block">
┌───┬───┬───┬───┬───┬───┬───┬───┬───┐
│ 1 │ 1 │ 1 │ 1 │ 1 │ 1 │ 1 │ 1 │ 1 │
└───┴───┴───┴───┴───┴───┴───┴───┴───┘
</pre>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<pre class="code python literal-block">
<span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">array</span><span class="punctuation">([</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">])</span>
</pre>
<pre class="code output literal-block">
┌───┬───┬───┬───┬───┬───┬───┬───┬───┐
│ 1 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 1 │ 0 │
└───┴───┴───┴───┴───┴───┴───┴───┴───┘
</pre>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<pre class="code python literal-block">
<span class="name">Z</span> <span class="operator">=</span> <span class="literal number integer">2</span><span class="operator">*</span><span class="name">np</span><span class="operator">.</span><span class="name">ones</span><span class="punctuation">(</span><span class="literal number integer">9</span><span class="punctuation">)</span>
</pre>
<pre class="code output literal-block">
┌───┬───┬───┬───┬───┬───┬───┬───┬───┐
│ 2 │ 2 │ 2 │ 2 │ 2 │ 2 │ 2 │ 2 │ 2 │
└───┴───┴───┴───┴───┴───┴───┴───┴───┘
</pre>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<pre class="code python literal-block">
<span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">arange</span><span class="punctuation">(</span><span class="literal number integer">9</span><span class="punctuation">)</span>
</pre>
<pre class="code output literal-block">
┌───┬───┬───┬───┬───┬───┬───┬───┬───┐
│ 0 │ 1 │ 2 │ 3 │ 4 │ 5 │ 6 │ 7 │ 8 │
└───┴───┴───┴───┴───┴───┴───┴───┴───┘
</pre>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<pre class="code python literal-block">
<span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">arange</span><span class="punctuation">(</span><span class="literal number integer">9</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="literal number integer">9</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">)</span>
</pre>
<pre class="code output literal-block">
┌───┐
│ 0 │
├───┤
│ 1 │
├───┤
│ 2 │
├───┤
│ 3 │
├───┤
│ 4 │
├───┤
│ 5 │
├───┤
│ 6 │
├───┤
│ 7 │
├───┤
│ 8 │
└───┘
</pre>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<pre class="code python literal-block">
<span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">arange</span><span class="punctuation">(</span><span class="literal number integer">9</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="literal number integer">3</span><span class="punctuation">,</span><span class="literal number integer">3</span><span class="punctuation">)</span>
</pre>
<pre class="code output literal-block">
┌───┬───┬───┐
│ 0 │ 1 │ 2 │
├───┼───┼───┤
│ 3 │ 4 │ 5 │
├───┼───┼───┤
│ 6 │ 7 │ 8 │
└───┴───┴───┘
</pre>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<pre class="code python literal-block">
<span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">random</span><span class="operator">.</span><span class="name">randint</span><span class="punctuation">(</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">9</span><span class="punctuation">,(</span><span class="literal number integer">3</span><span class="punctuation">,</span><span class="literal number integer">3</span><span class="punctuation">))</span>
</pre>
<pre class="code output literal-block">
┌───┬───┬───┐
│ 4 │ 5 │ 7 │
├───┼───┼───┤
│ 0 │ 2 │ 6 │
├───┼───┼───┤
│ 8 │ 4 │ 0 │
└───┴───┴───┘
</pre>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<pre class="code python literal-block">
<span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">linspace</span><span class="punctuation">(</span><span class="literal number integer">0</span><span class="punctuation">,</span> <span class="literal number integer">1</span><span class="punctuation">,</span> <span class="literal number integer">5</span><span class="punctuation">)</span>
</pre>
<pre class="code output literal-block">
┌──────┬──────┬──────┬──────┬──────┐
│ 0.00 │ 0.25 │ 0.50 │ 0.75 │ 1.00 │
└──────┴──────┴──────┴──────┴──────┘
</pre>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<pre class="code python literal-block">
<span class="name">np</span><span class="operator">.</span><span class="name">mgrid</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">:</span><span class="literal number integer">3</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">:</span><span class="literal number integer">3</span><span class="punctuation">]</span>
</pre>
<pre class="code output literal-block">
┌───┬───┬───┐   ┌───┬───┬───┐
│ 0 │ 0 │ 0 │   │ 0 │ 1 │ 2 │
├───┼───┼───┤   ├───┼───┼───┤
│ 1 │ 1 │ 1 │   │ 0 │ 1 │ 2 │
├───┼───┼───┤   ├───┼───┼───┤
│ 2 │ 2 │ 2 │   │ 0 │ 1 │ 2 │
└───┴───┴───┘   └───┴───┴───┘
</pre>
</div>
<div class="section" id="id60">
<h2><a class="toc-backref" href="#id57">Indexing</a></h2>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<pre class="code python literal-block">
<span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">arange</span><span class="punctuation">(</span><span class="literal number integer">9</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="literal number integer">3</span><span class="punctuation">,</span><span class="literal number integer">3</span><span class="punctuation">)</span>
<span class="name">Z</span><span class="punctuation">[</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">]</span>
</pre>
<pre class="code output literal-block">
┏━━━┓───┬───┐   ┏━━━┓
┃ 0 ┃ 1 │ 2 │ → ┃ 0 ┃ (scalar)
┗━━━┛───┼───┤   ┗━━━┛
│ 3 │ 4 │ 5 │
├───┼───┼───┤
│ 6 │ 7 │ 8 │
└───┴───┴───┘
</pre>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<pre class="code python literal-block">
<span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">arange</span><span class="punctuation">(</span><span class="literal number integer">9</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="literal number integer">3</span><span class="punctuation">,</span><span class="literal number integer">3</span><span class="punctuation">)</span>
<span class="name">Z</span><span class="punctuation">[</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">]</span>
</pre>
<pre class="code output literal-block">
┌───┬───┬───┐
│ 0 │ 1 │ 2 │
├───┼───┼───┤
│ 3 │ 4 │ 5 │
├───┼───┏━━━┓   ┏━━━┓
│ 6 │ 7 ┃ 8 ┃ → ┃ 8 ┃ (scalar)
└───┴───┗━━━┛   ┗━━━┛
</pre>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<pre class="code python literal-block">
<span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">arange</span><span class="punctuation">(</span><span class="literal number integer">9</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="literal number integer">3</span><span class="punctuation">,</span><span class="literal number integer">3</span><span class="punctuation">)</span>
<span class="name">Z</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">]</span>
</pre>
<pre class="code output literal-block">
┌───┬───┬───┐
│ 0 │ 1 │ 2 │
┏━━━┳━━━┳━━━┓   ┏━━━┳━━━┳━━━┓
┃ 3 ┃ 4 ┃ 5 ┃ → ┃ 3 ┃ 4 ┃ 5 ┃
┗━━━┻━━━┻━━━┛   ┗━━━┻━━━┻━━━┛
│ 6 │ 7 │ 8 │      (view)
└───┴───┴───┘
</pre>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<pre class="code python literal-block">
<span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">arange</span><span class="punctuation">(</span><span class="literal number integer">9</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="literal number integer">3</span><span class="punctuation">,</span><span class="literal number integer">3</span><span class="punctuation">)</span>
<span class="name">Z</span><span class="punctuation">[:,</span><span class="literal number integer">2</span><span class="punctuation">]</span>
</pre>
<pre class="code output literal-block">
┌───┬───┏━━━┓   ┏━━━┓
│ 0 │ 1 ┃ 2 ┃   ┃ 2 ┃
├───┼───┣━━━┫   ┣━━━┫
│ 3 │ 4 ┃ 5 ┃ → ┃ 5 ┃ (view)
├───┼───┣━━━┫   ┣━━━┫
│ 6 │ 7 ┃ 8 ┃   ┃ 8 ┃
└───┴───┗━━━┛   ┗━━━┛
</pre>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<pre class="code python literal-block">
<span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">arange</span><span class="punctuation">(</span><span class="literal number integer">9</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="literal number integer">3</span><span class="punctuation">,</span><span class="literal number integer">3</span><span class="punctuation">)</span>
<span class="name">Z</span><span class="punctuation">[</span><span class="literal number integer">1</span><span class="punctuation">:,</span><span class="literal number integer">1</span><span class="punctuation">:]</span>
</pre>
<pre class="code output literal-block">
┌───┬───┬───┐
│ 0 │ 1 │ 2 │    (view)
├───┏━━━┳━━━┓   ┏━━━┳━━━┓
│ 3 ┃ 4 ┃ 5 ┃   ┃ 4 ┃ 5 ┃
├───┣━━━╋━━━┫ → ┣━━━╋━━━┫
│ 6 ┃ 7 ┃ 8 ┃   ┃ 7 ┃ 8 ┃
└───┗━━━┻━━━┛   ┗━━━┻━━━┛
</pre>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<pre class="code python literal-block">
<span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">arange</span><span class="punctuation">(</span><span class="literal number integer">9</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="literal number integer">3</span><span class="punctuation">,</span><span class="literal number integer">3</span><span class="punctuation">)</span>
<span class="name">Z</span><span class="punctuation">[::</span><span class="literal number integer">2</span><span class="punctuation">,::</span><span class="literal number integer">2</span><span class="punctuation">]</span>
</pre>
<pre class="code output literal-block">
┏━━━┓───┏━━━┓   ┏━━━┳━━━┓
┃ 0 ┃ 1 ┃ 2 ┃   ┃ 0 ┃ 2 ┃
┗━━━┛───┗━━━┛ → ┣━━━╋━━━┫
│ 3 │ 4 │ 5 │   ┃ 6 ┃ 8 ┃
┏━━━┓───┏━━━┓   ┗━━━┻━━━┛
┃ 6 ┃ 7 ┃ 8 ┃    (view)
┗━━━┛───┗━━━┛
</pre>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<pre class="code python literal-block">
<span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">arange</span><span class="punctuation">(</span><span class="literal number integer">9</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="literal number integer">3</span><span class="punctuation">,</span><span class="literal number integer">3</span><span class="punctuation">)</span>
<span class="name">Z</span><span class="punctuation">[[</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">],[</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">2</span><span class="punctuation">]]</span>
</pre>
<pre class="code output literal-block">
┏━━━┓───┬───┐
┃ 0 ┃ 1 │ 2 │
┗━━━┛───┏━━━┓   ┏━━━┳━━━┓
│ 3 │ 4 ┃ 5 ┃ → ┃ 0 ┃ 5 ┃
├───┼───┗━━━┛   ┗━━━┻━━━┛
│ 6 │ 7 │ 8 │    (copy)
└───┴───┴───┘
</pre>
</div>
<div class="section" id="reshaping">
<h2><a class="toc-backref" href="#id57">Reshaping</a></h2>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<pre class="code python literal-block">
<span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">array</span><span class="punctuation">([</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">])</span>
</pre>
<pre class="code output literal-block">
┌───┬───┬───┬───┬───┬───┬───┬───┬───┬───┏━━━┓───┐
│ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 ┃ 1 ┃ 0 │
└───┴───┴───┴───┴───┴───┴───┴───┴───┴───┗━━━┛───┘
</pre>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<pre class="code python literal-block">
<span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">array</span><span class="punctuation">([</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">])</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="literal number integer">12</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">)</span>
</pre>
<pre class="code output literal-block">
┌───┐
│ 0 │
├───┤
│ 0 │
├───┤
│ 0 │
├───┤
│ 0 │
├───┤
│ 0 │
├───┤
│ 0 │
├───┤
│ 0 │
├───┤
│ 0 │
├───┤
│ 0 │
├───┤
│ 0 │
┏━━━┓
┃ 1 ┃
┗━━━┛
│ 0 │
└───┘
</pre>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<pre class="code python literal-block">
<span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">array</span><span class="punctuation">([</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">])</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="literal number integer">3</span><span class="punctuation">,</span><span class="literal number integer">4</span><span class="punctuation">)</span>
</pre>
<pre class="code output literal-block">
┌───┬───┬───┬───┐
│ 0 │ 0 │ 0 │ 0 │
├───┼───┼───┼───┤
│ 0 │ 0 │ 0 │ 0 │
├───┼───┏━━━┓───┤
│ 0 │ 0 ┃ 1 ┃ 0 │
└───┴───┗━━━┛───┘
</pre>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<pre class="code python literal-block">
<span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">array</span><span class="punctuation">([</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">])</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="literal number integer">4</span><span class="punctuation">,</span><span class="literal number integer">3</span><span class="punctuation">)</span>
</pre>
<pre class="code output literal-block">
┌───┬───┬───┐
│ 0 │ 0 │ 0 │
├───┼───┼───┤
│ 0 │ 0 │ 0 │
├───┼───┼───┤
│ 0 │ 0 │ 0 │
├───┏━━━┓───┤
│ 0 ┃ 1 ┃ 0 │
└───┗━━━┛───┘
</pre>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<pre class="code python literal-block">
<span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">array</span><span class="punctuation">([</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">])</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="literal number integer">6</span><span class="punctuation">,</span><span class="literal number integer">2</span><span class="punctuation">)</span>
</pre>
<pre class="code output literal-block">
┌───┬───┐
│ 0 │ 0 │
├───┼───┤
│ 0 │ 0 │
├───┼───┤
│ 0 │ 0 │
├───┼───┤
│ 0 │ 0 │
├───┼───┤
│ 0 │ 0 │
┏━━━┓───┤
┃ 1 ┃ 0 │
┗━━━┛───┘
</pre>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<pre class="code python literal-block">
<span class="name">Z</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">array</span><span class="punctuation">([</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">0</span><span class="punctuation">])</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="literal number integer">2</span><span class="punctuation">,</span><span class="literal number integer">6</span><span class="punctuation">)</span>
</pre>
<pre class="code output literal-block">
┌───┬───┬───┬───┬───┬───┐
│ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │
├───┼───┼───┼───┏━━━┓───┤
│ 0 │ 0 │ 0 │ 0 ┃ 1 ┃ 0 │
└───┴───┴───┴───┗━━━┛───┘
</pre>
</div>
<div class="section" id="broadcasting">
<h2><a class="toc-backref" href="#id57">Broadcasting</a></h2>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<pre class="code python literal-block">
<span class="name">Z1</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">arange</span><span class="punctuation">(</span><span class="literal number integer">9</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="literal number integer">3</span><span class="punctuation">,</span><span class="literal number integer">3</span><span class="punctuation">)</span>
<span class="name">Z2</span> <span class="operator">=</span> <span class="literal number integer">1</span>
<span class="name">Z1</span> <span class="operator">+</span> <span class="name">Z2</span>
</pre>
<pre class="code output literal-block">
┌───┬───┬───┐   ┌───┐   ┌───┬───┬───┐   ┏━━━┓───┬───┐   ┌───┬───┬───┐
│ 0 │ 1 │ 2 │ + │ 1 │ = │ 0 │ 1 │ 2 │ + ┃ 1 ┃ 1 │ 1 │ = │ 1 │ 2 │ 3 │
├───┼───┼───┤   └───┘   ├───┼───┼───┤   ┗━━━┛───┼───┤   ├───┼───┼───┤
│ 3 │ 4 │ 5 │           │ 3 │ 4 │ 5 │   │ 1 │ 1 │ 1 │   │ 4 │ 5 │ 6 │
├───┼───┼───┤           ├───┼───┼───┤   ├───┼───┼───┤   ├───┼───┼───┤
│ 6 │ 7 │ 8 │           │ 6 │ 7 │ 8 │   │ 1 │ 1 │ 1 │   │ 7 │ 8 │ 9 │
└───┴───┴───┘           └───┴───┴───┘   └───┴───┴───┘   └───┴───┴───┘
</pre>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<pre class="code python literal-block">
<span class="name">Z1</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">arange</span><span class="punctuation">(</span><span class="literal number integer">9</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="literal number integer">3</span><span class="punctuation">,</span><span class="literal number integer">3</span><span class="punctuation">)</span>
<span class="name">Z2</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">arange</span><span class="punctuation">(</span><span class="literal number integer">3</span><span class="punctuation">)[::</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">]</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="literal number integer">3</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">)</span>
<span class="name">Z1</span> <span class="operator">+</span> <span class="name">Z2</span>
</pre>
<pre class="code output literal-block">
┌───┬───┬───┐   ┌───┐   ┌───┬───┬───┐   ┏━━━┓───┬───┐   ┌───┬───┬───┐
│ 0 │ 1 │ 2 │ + │ 2 │ = │ 0 │ 1 │ 2 │ + ┃ 2 ┃ 2 │ 2 │ = │ 2 │ 3 │ 4 │
├───┼───┼───┤   ├───┤   ├───┼───┼───┤   ┣━━━┫───┼───┤   ├───┼───┼───┤
│ 3 │ 4 │ 5 │   │ 1 │   │ 3 │ 4 │ 5 │   ┃ 1 ┃ 1 │ 1 │   │ 4 │ 5 │ 6 │
├───┼───┼───┤   ├───┤   ├───┼───┼───┤   ┣━━━┫───┼───┤   ├───┼───┼───┤
│ 6 │ 7 │ 8 │   │ 0 │   │ 6 │ 7 │ 8 │   ┃ 0 ┃ 0 │ 0 │   │ 6 │ 7 │ 8 │
└───┴───┴───┘   └───┘   └───┴───┴───┘   ┗━━━┛───┴───┘   └───┴───┴───┘
</pre>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<pre class="code python literal-block">
<span class="name">Z1</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">arange</span><span class="punctuation">(</span><span class="literal number integer">9</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="literal number integer">3</span><span class="punctuation">,</span><span class="literal number integer">3</span><span class="punctuation">)</span>
<span class="name">Z2</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">arange</span><span class="punctuation">(</span><span class="literal number integer">3</span><span class="punctuation">)[::</span><span class="operator">-</span><span class="literal number integer">1</span><span class="punctuation">]</span>
<span class="name">Z1</span> <span class="operator">+</span> <span class="name">Z2</span>
</pre>
<pre class="code output literal-block">
┌───┬───┬───┐   ┌───┬───┬───┐   ┌───┬───┬───┐   ┏━━━┳━━━┳━━━┓   ┌───┬───┬───┐
│ 0 │ 1 │ 2 │ + │ 2 │ 1 │ 0 │ = │ 0 │ 1 │ 2 │ + ┃ 2 ┃ 1 ┃ 0 ┃ = │ 2 │ 2 │ 2 │
├───┼───┼───┤   └───┴───┴───┘   ├───┼───┼───┤   ┗━━━┻━━━┻━━━┛   ├───┼───┼───┤
│ 3 │ 4 │ 5 │                   │ 3 │ 4 │ 5 │   │ 2 │ 1 │ 0 │   │ 5 │ 5 │ 5 │
├───┼───┼───┤                   ├───┼───┼───┤   ├───┼───┼───┤   ├───┼───┼───┤
│ 6 │ 7 │ 8 │                   │ 6 │ 7 │ 8 │   │ 2 │ 1 │ 0 │   │ 8 │ 8 │ 8 │
└───┴───┴───┘                   └───┴───┴───┘   └───┴───┴───┘   └───┴───┴───┘
</pre>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<pre class="code python literal-block">
<span class="name">Z1</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">arange</span><span class="punctuation">(</span><span class="literal number integer">3</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="literal number integer">3</span><span class="punctuation">,</span><span class="literal number integer">1</span><span class="punctuation">)</span>
<span class="name">Z2</span> <span class="operator">=</span> <span class="name">np</span><span class="operator">.</span><span class="name">arange</span><span class="punctuation">(</span><span class="literal number integer">3</span><span class="punctuation">)</span><span class="operator">.</span><span class="name">reshape</span><span class="punctuation">(</span><span class="literal number integer">1</span><span class="punctuation">,</span><span class="literal number integer">3</span><span class="punctuation">)</span>
<span class="name">Z1</span> <span class="operator">+</span> <span class="name">Z2</span>
</pre>
<pre class="code output literal-block">
┌───┐   ┌───┬───┬───┐   ┏━━━┓───┬───┐   ┏━━━┳━━━┳━━━┓   ┌───┬───┬───┐
│ 0 │ + │ 0 │ 1 │ 2 │ = ┃ 0 ┃ 0 │ 0 │ + ┃ 0 ┃ 1 ┃ 2 ┃ = │ 0 │ 1 │ 2 │
├───┤   └───┴───┴───┘   ┣━━━┫───┼───┤   ┗━━━┻━━━┻━━━┛   ├───┼───┼───┤
│ 1 │                   ┃ 1 ┃ 1 │ 1 │   │ 0 │ 1 │ 2 │   │ 1 │ 2 │ 3 │
├───┤                   ┣━━━┫───┼───┤   ├───┼───┼───┤   ├───┼───┼───┤
│ 2 │                   ┃ 2 ┃ 2 │ 2 │   │ 0 │ 1 │ 2 │   │ 2 │ 3 │ 4 │
└───┘                   ┗━━━┛───┴───┘   └───┴───┴───┘   └───┴───┴───┘
</pre>
</div>
</div>
<div class="section" id="bibliography">
<h1><a class="toc-backref" href="#table-of-contents">Bibliography</a></h1>
<p>This is a curated list of some NumPy related resources (articles, books &amp;
tutorials) addressing different aspects of NumPy. Some are very specific to
NumPy/Scipy while some others offer a broader view on numerical computing.</p>
<div class="contents local topic" id="id61">
<p class="topic-title first"><strong>Contents</strong></p>
<ul class="simple">
<li><a class="reference internal" href="#tutorials" id="id199">Tutorials</a></li>
<li><a class="reference internal" href="#articles" id="id200">Articles</a></li>
<li><a class="reference internal" href="#books" id="id201">Books</a></li>
</ul>
</div>
<div class="section" id="tutorials">
<h2><a class="toc-backref" href="#id61">Tutorials</a></h2>
<ul class="simple">
<li><a class="reference external" href="http://www.labri.fr/perso/nrougier/teaching/numpy.100/index.html">100 Numpy exercises</a>, Nicolas P. Rougier, 2016.</li>
<li><a class="reference external" href="http://www.labri.fr/perso/nrougier/teaching/numpy/numpy.html">Numpy tutorial</a>, Nicolas P. Rougier, 2015.</li>
<li><a class="reference external" href="http://www.python-course.eu/numpy.php">Python course</a>, Bernd Klein, 2015.</li>
<li><a class="reference external" href="https://engineering.ucsb.edu/~shell/che210d/numpy.pdf">An introduction to Numpy and Scipy</a>, M. Scott Shell, 2014.</li>
<li><a class="reference external" href="http://cs231n.github.io/python-numpy-tutorial/">Python Numpy tutorial</a>, Justin Johnson, 2014.</li>
<li><a class="reference external" href="https://docs.scipy.org/doc/numpy/user/quickstart.html">Quickstart tutorial</a>, Numpy developers, 2009.</li>
<li><a class="reference external" href="http://mentat.za.net/numpy/numpy_advanced_slides/">Numpy medkits</a>, Stéfan van der Walt, 2008.</li>
</ul>
</div>
<div class="section" id="articles">
<h2><a class="toc-backref" href="#id61">Articles</a></h2>
<ul>
<li><div class="first line-block">
<div class="line"><a class="reference external" href="https://hal.inria.fr/inria-00564007/document">The NumPy array: a structure for efficient numerical computation</a></div>
<div class="line">Stéfan van der Walt, Chris Colbert &amp; Gael Varoquaux,
Computing in Science and Engineering, 13(2), 2011.</div>
</div>
<div class="abstract docutils container">
<p>In the Python world, NumPy arrays are the standard representation for
numerical data and enable efficient implementation of numerical
computations in a high-level language. As this effort shows, NumPy
performance can be improved through three techniques: vectorizing
calculations, avoiding copying data in memory, and minimizing operation
counts.</p>
</div>
</li>
<li><div class="first line-block">
<div class="line"><a class="reference external" href="http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.397.6097">Vectorised algorithms for spiking neural network simulation</a></div>
<div class="line">Romain Brette &amp; Dan F. M. Goodman,
Neural Computation, 23(6), 2010.</div>
</div>
<div class="abstract docutils container">
<p>High-level languages (Matlab, Python) are popular in neuroscience because
they are flexible and accelerate development. However, for simulating
spiking neural networks, the cost of interpretation is a bottleneck. We
describe a set of algorithms to simulate large spiking neural networks
efficiently with high-level languages using vector-based operations. These
algorithms constitute the core of Brian, a spiking neural network
simulator written in the Python language. Vectorized simulation makes it
possible to combine the flexibility of high-level languages with the
computational efficiency usually associated with compiled languages.</p>
</div>
</li>
<li><div class="first line-block">
<div class="line"><a class="reference external" href="http://dl.acm.org/citation.cfm?id=1251830">Python for Scientific Computing</a></div>
<div class="line">Travis E. Oliphant,
Computing in Science &amp; Engineering, 9(3), 2007.</div>
</div>
<div class="abstract docutils container">
<p>By itself, Python is an excellent &quot;steering&quot; language for scientific codes
written in other languages. However, with additional basic tools, Python
transforms into a high-level language suited for scientific and
engineering code that's often fast enough to be immediately useful but
also flexible enough to be sped up with additional extensions.</p>
</div>
</li>
</ul>
</div>
<div class="section" id="books">
<h2><a class="toc-backref" href="#id61">Books</a></h2>
<ul>
<li><div class="first line-block">
<div class="line"><a class="reference external" href="http://www.scipy-lectures.org">SciPy Lecture Notes</a>,</div>
<div class="line">Gaël Varoquaux, Emmanuelle Gouillart, Olav Vahtras et al., 2016.</div>
</div>
<div class="abstract docutils container">
<p>One document to learn numerics, science, and data with Python.  Tutorials
on the scientific Python ecosystem: a quick introduction to central tools
and techniques. The different chapters each correspond to a 1 to 2 hours
course with increasing level of expertise, from beginner to expert.</p>
</div>
</li>
<li><div class="first line-block">
<div class="line"><a class="reference external" href="http://shop.oreilly.com/product/0636920034919.do">Python Data Science Handbook</a></div>
<div class="line">Jake van der Plas, O'Reilly, 2016.</div>
</div>
<div class="abstract docutils container">
<p>The Python Data Science Handbook provides a reference to the breadth of
computational and statistical methods that are central to data—intensive
science, research, and discovery. People with a programming background who
want to use Python effectively for data science tasks will learn how to
face a variety of problems: for example, how can you read this data format
into your script? How can you manipulate, transform, and clean this data?
How can you use this data to gain insight, answer questions, or to build
statistical or machine learning models?</p>
</div>
</li>
<li><div class="first line-block">
<div class="line"><a class="reference external" href="http://shop.oreilly.com/product/0636920038481.do">Elegant SciPy: The Art of Scientific Python</a></div>
<div class="line">Juan Nunez-Iglesias, Stéfan van der Walt, Harriet Dashnow, O'Reilly, 2016.</div>
</div>
<div class="abstract docutils container">
<p>Welcome to Scientific Python and its community! With this practical book,
you'll learn the fundamental parts of SciPy and related libraries, and get
a taste of beautiful, easy-to-read code that you can use in practice. More
and more scientists are programming, and the SciPy library is here to
help.  Finding useful functions and using them correctly, efficiently, and
in easily readable code are two very different things. You'll learn by
example with some of the best code available, selected to cover a wide
range of SciPy and related libraries—including scikit-learn, scikit-image,
toolz, and pandas.</p>
</div>
</li>
<li><div class="first line-block">
<div class="line"><a class="reference external" href="https://www.packtpub.com/big-data-and-business-intelligence/learning-ipython-interactive-computing-and-data-visualization-sec">Learning IPython for Interactive Computing and Data Visualization</a></div>
<div class="line">Cyrille Rossant, Packt Publishing, 2015.</div>
</div>
<div class="abstract docutils container">
<p>This book is a beginner-friendly guide to the Python data analysis
platform. After an introduction to the Python language, IPython, and the
Jupyter Notebook, you will learn how to analyze and visualize data on
real-world examples, how to create graphical user interfaces for image
processing in the Notebook, and how to perform fast numerical computations
for scientific simulations with NumPy, Numba, Cython, and ipyparallel. By
the end of this book, you will be able to perform in-depth analyses of all
sorts of data.</p>
</div>
</li>
<li><div class="first line-block">
<div class="line"><a class="reference external" href="https://www.safaribooksonline.com/library/view/scipy-and-numpy/9781449361600/">SciPy and NumPy</a></div>
<div class="line">Eli Bressert, O'Reilly Media, Inc., 2012</div>
</div>
<div class="abstract docutils container">
<p>Are you new to SciPy and NumPy? Do you want to learn it quickly and easily
through examples and concise introduction? Then this is the book for
you. You’ll cut through the complexity of online documentation and
discover how easily you can get up to speed with these Python libraries.</p>
</div>
</li>
<li><div class="first line-block">
<div class="line"><a class="reference external" href="http://shop.oreilly.com/product/0636920023784.do">Python for Data Analysis</a></div>
<div class="line">Wes McKinney, O'Reilly Media, Inc., 2012.</div>
</div>
<div class="abstract docutils container">
<p>Looking for complete instructions on manipulating, processing, cleaning,
and crunching structured data in Python? This hands-on book is packed
with practical cases studies that show you how to effectively solve a
broad set of data analysis problems, using several Python libraries.*</p>
</div>
</li>
<li><div class="first line-block">
<div class="line"><a class="reference external" href="http://csc.ucdavis.edu/~chaos/courses/nlp/Software/NumPyBook.pdf">Guide to NumPy</a></div>
<div class="line">Travis Oliphant, 2006</div>
</div>
<div class="abstract docutils container">
<p>This book only briefly outlines some of the infrastructure that surrounds
the basic objects in NumPy to provide the additional functionality
contained in the older Numeric package (i.e. LinearAlgebra, RandomArray,
FFT). This infrastructure in NumPy includes basic linear algebra routines,
Fourier transform capabilities, and random number generators. In addition,
the f2py module is described in its own documentation, and so is only
briefly mentioned in the second part of the book.</p>
</div>
</li>
</ul>
<!-- - - - Compilation - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<!-- rst2html.py - -link-stylesheet - -stylesheet=markdown.css book.rst book.html -->
</div>
</div>
</div>
</body>
</html>
